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ABSTRACT 


Timeliness is paramount when restoring partial airfield capability after an airfield 
has been damaged. The project’s focus was to develop conceptual system alternatives for 
improving the timeliness of airfield damage assessment. The systems engineering 
methods used included a morphological box and Pugh matrix for determining physical 
components and Constructive Systems Engineering Cost Model for cost analysis. 

Two separate improved airfield damage assessment system solutions were 
designed, evaluated, and compared regarding their cost and performance. Equipment and 
standard operating procedures selected were based on the design reference mission 
(DRM) and the limited time to complete the study. The first system used a remotely 
piloted aircraft (RPA) paired with a day camera. The second system used a set of fixed- 
tower emplacements, each with a day camera. Models were created and simulations were 
executed against the DRM to demonstrate the performance for each alternative. After 
reviewing the cost and simulation data, the RPA alternative showed superior 
performance. The modular design could be used with other airfield configurations. The 
RPA alternative cost more than the fixed-tower alternative. Eurther research is 
recommended in order to determine the cost and performance improvements that might 
result from different equipment configurations and improved camera technology. 
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EXECUTIVE SUMMARY 


The United States Air Force Emergency Operations Center (EOC) has the 
responsibility to assess the extent of airfield damage after an airfield has been damaged. 
The Airfield Damage Assessment (ADA) process must be accurate and timely in 
measuring airfield damage. Based upon past airfield damage repair (ADR) experience, 
the assessment phase can be quite time-consuming and dangerous. The damage 
assessment is conducted in two phases: initial reconnaissance and detailed damage 
assessment. The initial reconnaissance provides enough information to allow the EOC to 
send ADA teams to parts of the airfield that need a more detailed assessment. The 
detailed damage assessment provides detailed reporting of airfield runway damage and 
identification of unexploded ordnance (UXO). The completion time is highly variable 
based on the quantity and types of UXOs and the extent of runway damage. 

The problem is that the current ADA process, when UXO is present, takes 
significantly longer than is required to support timely ADR. The Department of Defense 
is looking for a solution that will provide accurate location information for damage and 
UXOs, which reduces the time to complete the ADA mission, along with complete 
coverage of the damaged area to be assessed, within the specified timeframe of 
45 minutes. 

There are currently no fielded autonomous systems capable of performing the role 
of ADA. The benefit of this project was to provide the stakeholders with sufficient 
information to understand the benefits and costs associated with the original process and 
two alternative systems. A systems engineering (SE) analysis has been performed 
comparing the current process to the potential autonomous system solutions being 
researched. 

An analysis of alternatives was documented, focusing on two alternative systems 
that have the potential to meet the criteria to perform ADA. Because the purpose of this 
project was to focus on the ADA, the Design Reference Mission (DRM) was defined to 
begin after an enemy attack on a U.S. airfield. 
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The scope of this project was to focus solely on the airfield damage assessment 
activity. The scope of the problem being researched includes the activities that begin after 
the airfield attack and initial reconnaissance for damage and ended once the damage 
assessment results were communicated to the EOC. 

This alternative assessment is provided for the stakeholders to have a high-level 
view of the performance differences between the three systems performing ADA 
activities. The merits of each system can be evaluated against each other and the metrics 
used to determine “success” against a known DRM. In order to make as complete a 
comparison as possible, focus was placed on the development costs (software, SE, and 
10-year maintenance), and instantiation costs (hardware, installation, and 10-year 
maintenance), which could be readily accessed in the timeframe for this report. 

Eor this project, the system requirements were refined based on stakeholder 
needs. The stakeholder needs for the new improved airfield damage assessment system 
(lADAS) were identified based on user representatives, the problem statement, literature 
research, and stakeholder analysis. A functional architecture was generated for the 
current airfield damage assessment system (ADAS), lADAS I, and lADAS II. This top- 
down decomposition showed the functions that were performed for a notional ADA 
mission. 

The process of determining the potential alternatives for the lADAS system 
included determining the components used in the current ADAS system, and those 
potential components that could be leveraged for the lADAS system. The SE tools used 
to analyze the necessary ADAS mission components were the morphological box and the 
Pugh matrix SE concepts. Based on the analysis of alternatives results, the lADAS I 
system analyzed in the project was a remote piloted aircraft (RPA) with a day camera and 
wireless sensors. The lADAS II system analyzed was a network of stationary towers each 
with a day camera and wireless sensor. 

The functional analysis of these alternatives included the creation of multiple 
types of diagrams to be used as tools to fully understand the functional capabilities of the 
system. This analysis included the use of the functional architecture hierarchy chart. 


XX 



functional block diagram, Integrated Computer Aided Manufacturing Definition for 
Function Modeling (IDEFO) diagrams, sequence diagrams, class diagrams, and concept 
of operations diagram. 

In order to best determine a viable alternative, the decomposed architectures were 
implemented into modeling and simulation tools. Imagine That Inc. Extendsim software 
was used to conduct this analysis. By developing the alternatives into models, the 
processes were able to be simulated and repeated in order to provide measurement data 
for analysis and evaluation. The models were run 500 times for statistical significance 
and to also model system variability. The model was built to allow for the input of six 
different types of damage and three types of UXO. The probability of detection, 
classification time, and measurement time for each damage and UXO type were 
estimated based on input from subject-matter experts and engineering judgment. 

Several measures were derived from the simulation model to evaluate the 
effectiveness of the alternatives. The selected measures used throughout the simulation 
and analysis are percent airfield damage assessed, percent UXO assessed, airfield 
damage assessment time, travel/detection time, classification time, and communication 
time. 

Einally, the life-cycle cost components for each lADAS alternative were as 
follows: research and development, SE, personnel, and operations and support costs 
(hardware, training, and 10-year maintenance). The SE cost was determined using 
Constructive Systems Engineering Cost Model (COSYSMO). The total life-cycle costs 
were estimated to be: 

ADAS $1782K 

lADAS I $2944K 

lADAS II $I436K 

By comparing the system alternatives to the measures of effectiveness associated 
with this projeet, lADAS I was the recommended solution for the stakeholders. The 
lADAS I system met, or exceeded, the threshold values for both assessing the percent of 



damage and UXO mission parameters. In addition, the lADAS I system significantly 
reduced the ADA timeline. The overall capability of the system delivers significantly 
reduced ADA time as compared to the original ADAS. For the DRM scenario studied in 
this project, the current ADAT time was estimated at 174 minutes. The lADAS I 
completed the simulation in just 52 minutes. The implementation cost for lADAS I was 
higher than the lADAS II alternative; however, the lADAS I system conducted the 
mission in a more efficient and cost-effective manner. The lADAS I has a significantly 
smaller footprint on the airfield of interest. This would include a small hardened storage 
container for the RPA and spares, along with the ground control station. Additionally, 
lADAS I has an added feature of being easily moved from one airfield to another as 
required. The lADAS II has a more significant logistics impact on the infrastructure of 
the airfield, having towers placed at fixed intervals along the area of interest, as defined 
in the DRM. 

Considering the multitude of options available for outfitting an lADAS, the 
project focused on defined alternatives due to the limited timeframe available. The DRM 
allows for many alternative solutions including a variety of RPA options, tower elements, 
cameras, and sensors. Future systems under consideration can take advantage of these 
options as well as upgraded benchmarks for key system elements such as image quality, 
computer processing speed, and networked communications. As noted in the 
computations for lADAS I and lADAS II, communications and processing were 
significant contributors to the time spent on the overall mission. 
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I. 


INTRODUCTION 


The Department of Defense (DOD) operates in a multitude of environments, 
including homeland security, training, peace-keeping, and combat. In all of these 
environments, especially in combat. United States Armed Forces must be able to conduct 
aircraft operations on a variety of airfield types and locations. Anytime an airfield is 
damaged through natural disasters, accidents, or hostile means, the ability to launch and 
recover aircraft sorties is diminished. In order to restore full airfield capability, a series of 
steps must be taken. One of the first steps is to complete an airfield damage assessment 
(ADA). 

A. BACKGROUND 

According to the document. Airfield Damage Assessment after Attack Tactics, 
Techniques, and Procedures (TTP) (Department of the Air Force 2016), “Airfield 
damage assessment is the process of locating, classifying, and measuring airfield damage 
and unexploded ordnance (UXO) after an attack.” The current process is very personnel¬ 
intensive with little automation. 

The United States Air Force (USAF) Emergency Operations Center (EOC) has 
the responsibility to assess the extent of airfield damage after an airfield has been 
damaged. At least three minimum airfield operating surface (MAOS) candidates are 
selected to be briefed to the Installation Commander as soon as possible after the attack 
(Department of the Air Eorce 2016). To help meet that operational requirement, the 
airfield damage assessment (ADA) process must be accurate and timely in measuring 
airfield damage. Based upon past airfield damage repair (ADR) experience, the 
assessment phase can be quite time consuming and dangerous. The damage assessment is 
conducted in two phases: initial reconnaissance and detailed damage assessment. The 
initial reconnaissance rapidly assesses damage to broad areas of the airfield pavement 
from a distance and is done fairly quickly (estimated 10 to 20 minutes), but does not 
provide a detailed damage estimate as observations are done at a long distance from the 
damaged airfield areas. The initial reconnaissance provides enough information to allow 
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the EOC to send airfield damage assessment teams (ADAT) to parts of the airfield that 
need a more detailed assessment (Department of the Air Force 2016). The detailed 
damage assessment provides detailed reporting of airfield runway damage and 
identification of UXO. The completion time is highly variable based on the quantity and 
types of UXOs and the extent of runway damage. 

According to the AFP AM 10-219, Contingency and Disaster Planning 
(Department of the Air Force 2008), managing airfield damage consists of: 

1. Pre-positioning of Assets 

2. Reporting of Airfield Damage Event 

3. Initiating an ADA 

4. Determining the MAOS 

5. Performing Rapid Runway Repair 

The preparation step (#1, in the list provided above), especially in an 
expeditionary (hostile) environment, usually occurs well ahead of any engagement by an 
enemy. Resources, such as personnel, heavy equipment, and repair materials are brought 
to strategic airfields in preparation for being able to respond to any situation, which could 
interrupt the mission of the airbase. Depending on the characteristics of the airbase 
(number of runways, supported aircraft, and criticality of the missions being supported), 
this may involve a considerable amount of resources. 

The first step in the actual scenario (#2, in the list provided above) occurs when 
an event creates some type of damage hazard to the airfield, or surrounding 
infrastructure, necessary to flight operations. This may include bombing runs from enemy 
aircraft, sabotage from insurgent forces, accidents/crashes by friendly aircraft, or damage 
from natural disasters, which could hinder the mission of the airfield. 

This triggers the next step (#3, in the list provided above) in the scenario, which is 
assessing the airfield damage. The airfield damage assessment teams are typically 
comprised of five to seven individuals and are sent out to gather the information 
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necessary to determine the quantity and location of damage and UXOs within the 
boundaries of the airfield (Mallerski 2009). The number of ADATs required for an 
operation is based on a number of factors such as “the number of runways and airfield 
operating surfaces that need to be maintained” (Department of the Navy 2001). Those 
teams first have to wait for the all-clear to be given, which is normally provided at the 
end of the air raid, or conclusion of the storm. Either event would trigger a need for an 
assessment of damage. The ADATs, using armored vehicles or on foot, travel 
predetermined routes and inspect for potential damage to various airfield infrastructures, 
runways, taxiways, and apron surfaces. The ADATs are usually comprised of a team 
leader, two explosive ordnance disposal technicians, one radio operator/driver, one spall 
damage assessor, and two crater damage assessors (Department of the Navy 2001). See 
Table 1 for damage definitions. The ADATs then perform their role of mapping the 
damage sites (size and location) and UXO locations. Currently, all of the steps within the 
detailed assessment portion are performed manually. The team gets into position, 
performs their operations, and relays the information to the EOC. 

The information from the ADATs is transferred to the MAOS Selection Cell (#4, 
in the list provided above). This current process is dangerous to personnel as UXO or 
time delayed munitions may be present on the routes they travel. Additionally, this 
process takes up valuable time and could easily lead to miscommunication. The MAOS 
Selection Cell’s purpose is to calculate the minimum airfield, which could sustain 
operations, or determine the minimum amount of repair work, which could bring about 
the minimum airfield necessary. The MAOS Selection Cell manually enters the reported 
damage into the Geospatial Expeditionary Planning Tool (GeoExPT) system (a U.S. 
Government off-the shelf product). According to GeoExPT’s website, GeoExPT “is a 
decision support tool for mission planners and engineers to ... analyze and repair airfield 
damage for optimal selection” of the MAOS (Dynamic Software Solutions 2017). The 
MAOS Selection Cell then selects different MAOS options in order to bring the airfield 
back into operation. According to Air Force Tactics, Techniques, and Procedures 3- 
23.11, “the MAOS Selection Team briefs the MAOS candidates by order of preference to 
the Installation Commander or Senior Airfield Authority who then selects the preferred 
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MAOS that must be cleared and repaired to launch and recover aircraft” (Department of 
the Air Force 2016). The MAOS is a section of the airfield that can be operated in 
isolation from the rest, making it a priority for repairs. The time required to complete the 
ADA is heavily dependent on the amount of damage and number of teams available to 
survey the required areas. 

The final step (#5, in the list provided above) in the process is the actual repair of 
the runway and removal of the UXO, if present. Personnel would draw upon the pre¬ 
positioned equipment and material to go about the work to re-establish flight operations 
following the activity, which impeded them in the first place. Any damage not listed in 
Table 1 is not within the scope of this project. 

Performing ADA is fundamental to resuming activities. As such, “speed and 
accuracy during damage assessment are essential for the success of subsequent rapid 
airfield damage repair activities.” (Department of the Air Force 2016). The USAF has an 
“ongoing project,” named Rapid Airfield Damage Assessment System (RADAS), to 
perform the damage assessment using remote sensing techniques in order to accomplish 
the task quicker and in a safer environment (Earth Imaging Journal 2015). A mixture of 
ground mobile systems, fixed-installation systems (tower-based), and unmanned aircraft 
systems technology has been examined. Listed below are some of the technologies that 
have been explored to date. 

1. “Idaho National Laboratory (INL) developed an unmanned aerial vehicle 
(UAV) based system, RADAS, for rapid airfield damage assessment. These 
operations are usually conducted by two, three-man teams navigating the field 
in vehicles and require between 60 to 90 minutes to complete” (Satnews Daily 
2009). NOTE: According to the USAL, the terminology used for a UAV is a 
remotely piloted aircraft (RPA). Depending on the source of the information, 
this terminology may be used interchangeably. 

2. iLerret and Super Bullseye systems designed by Stratech Systems Limited 
were also being integrated into RADAS. The Super Bullseye sensors were 
placed in fixed positions to detect weapon impact times and locations and the 
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iFerret sensors can scan the runway to assess damage in real time (Echerri 
2015). 

3. The U.S. Army is investigating methods for airfield assessments. The U.S. 
Army’s Common Robotic System - Individual program is “to provide 
dismounted troops with the ability to conduct lower-level reconnaissance, 
surveillance, and target acquisition; and to remotely perform chemical, 
biological, radiological and nuclear detection” (Tomkins 2016). Additional 
features such as remote UXO disposal and counter measure operations were 
also considered (Tomkins 2016). 

4. Hydra Fusion Tools by Lockheed Martin CDL Systems is developing a near 
real-time software solution to generate a three dimensional model from data 
collected on Lockheed Martin’s Indago quadcopter using the Snap Dragon 12 
megapixel camera. This product can be deployed using laptop systems. 
(Chandler 2016) The concept is to deliver accurate images of the damage to 
decrease potential personnel damage as well as decrease response time. 


5 



Table 1. ADR Damage Descriptions 


Type of 
Damage 

Craters 


Description 


Graphical Description 


“Left when an object punctures the 
bottom surface of the pavement and 
aggregate is exposed. They can be as 
small as three feet and as large as 50 
feet” (Earth Imaging Journal 2015). 



Spall 


“Similar to a crater, but it does not 
puncture the bottom surface of the 
pavement” (Earth Imaging Journal 
2015). 


Source: (Eiller2014) 



Source: (Eiller 2014) 


Continued on next page. 
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Table 1. Continued from previous page. 


Type of 
Damage 

Camouflet 


Description 


Graphical Description 


“Munition penetrates the pavement and 
explodes under the surface to create a 
cavity. These are difficult to find and a 
dangerous hazard as aircraft weight 
can collapse these unseen holes” (Earth 
Imaging Journal 2015). 



UXO 


“The main challenge is to deter mi ne 
the classification (e.g., bomb, missile, 
or rocket), but sensors need to be able 
to determine the fuse type to assess the 
threat as well as the mitigation 
strategy” (Earth Imaging Journal 
2015). 


Source: (Eiller 2014) 



Source: (Eiller 2014) 
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According to an article in the Earth Imaging Journal, the RADAS Government 
Program Lead, USAF, stated that the goal was to accomplish the damage assessment 
within 30 minutes with a “24/7 capability that can be used in all weather and in a variety 
of conditions” (Earth Imaging Journal 2015). Air Force engineers have looked at these 
imaging sensors in support of RADAS: 

• video 

• electro-optical 

• infrared (long wave, medium wave infrared, near infrared, shortwave 

infrared) 

• radio detection and ranging (RADAR) (millimeter wave, Ku, X-Band, 

synthetic aperture RADAR) 

• seismic and acoustic 

“An important consideration when choosing sensors for ADR is being able to 
recognize the different types of airfield damage” (Earth Imaging Journal 2015) as 
described in Table 1. 

To date, technology limitations have posed some issues. “Electro-optical sensors 
work well in a daytime environment and in all weather conditions, but not at night. 
Infrared works well at night, but small amounts of weather create large problems. [The] 
RADAR is another platform that performs well in most environments, but does very poor 
during rain or bad weather” (Earth Imaging Journal 2015). 

B. PROBLEM STATEMENT 

The problem is that the current ADA process, when UXO is present, takes 
significantly longer than is required to support timely airfield damage repair. The DOD is 
looking for a solution that will provide accurate location information for damage and 
UXOs, along with complete coverage of the damaged area to be assessed. 

C. BENEFIT OF STUDY 

There are no fielded autonomous systems capable of performing the role of ADA, 
although there are several prototype systems that had the potential for performing that 
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role. The benefits of this project were to provide the necessary stakeholders with 
sufficient information to understand the benefits and costs associated with two alternative 
systems. A systems engineering (SE) analysis has been performed comparing the current 
process to the potential autonomous systems being researched. 

An SE analysis of alternatives (AoA) was documented, focusing on two 
alternative systems that had the potential to meet the criteria to perform ADA. 
Requirements were documented. Models were built and simulations were run to estimate 
how effective those systems operate. The quantitative analysis showed the results against 
the baseline of having personnel perform 100% of the operations. 

D. PROJECT SCOPE 

1. IN SCOPE 

The scope of this project was focused solely on the airfield damage assessment 
activity (Eigure 1). The scope of the problem being research includes the activities that 
begin after the airfield attack and initial reconnaissance for damage. The scope ended 
once the damage assessment results were communicated to the EOC. The ADA activity 
was in the DOD context. The scope of the analysis was narrowed to certain listed 
requirements as covered in the design reference mission (DRM). 
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As shown in the figure, there are many steps involved with returning an airfield back to 
an operational status. The focus of this project was limited to performing the damage 
assessment. 


Figure 1. Scope of Project 


The architecture and cost analysis of the current system were described. Two 
alternative system solutions were selected and their architecture and cost analysis were 
described. An AoA was performed and the results were documented in this report. In the 
end, the project stakeholders have both quantitative and qualitative analyses in order to 
compare each solution against one another. 

2. OUT OF SCOPE 

The activities related to determining the MAOS and performing runway repair 
were out of the scope of this project. Any damage not listed in Table 1 was considered 
debris and therefore, not within the scope of this project. The ADA activity for this report 
does not cover airfield damage caused by natural disasters. 

E. PROJECT OBJECTIVES 

The project objectives were to provide: 

1. A description of the current baseline ADAS. 
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2. Descriptions of two new conceptual systems (Improved Airfield Damage 
Assessment System (lADAS) I and lADAS II). 

3. A cost-effectiveness comparison of the baseline ADAS and the two 
conceptual lADAS systems. 

4. A discussion on recommendations about what the next steps should be. 

F. STAKEHOLDER IDENTIFICATION AND ANALYSIS 

The identified stakeholders are the EOC, the U.S. Air Force Civil Engineer Center 
(AFCEC) and Readiness Directorate, the U.S. Air Force Rapid Engineer Deployable 
Heavy Operational Repair Squadron Engineer (RED HORSE), and the U.S. Navy 
SeaBees. 

1. EOC 

The EOC is responsible for coordinating the airfield recovery process to include 
the ADR. In order to develop an airfield recovery plan, the EOC must first collect 
damage assessments of the take-off and landing surfaces as well as hazards that could 
impede the recovery process (such as UXOs, and damage to the airbase that could 
prevent airfield recovery efforts). By providing the EOC accurate near-real time ADA, 
the EOC will be able to reduce the time between the attack and the creation of the MAOS 
and allow the ADR teams to be released and begin airfield repair when the base is sent 
into alarm black/initial release. Alarm black/initial release is the state of an airbase after 
an attack has been completed. The EOC announces the condition alarm black/initial 
release when it is time to send the AD AT out to complete their mission. 

2. AFCEC and READINESS DIRECTORATE 

From the AFCEC/Readiness website (U.S. Air Force Civil Engineer Center 
2017), their mission is described below: 

The Readiness Directorate, located at Tyndall Air Force Base, Florida, 
provides readiness and emergency services support and serves as the 
source for civil engineer research, development and acquisition to the Air 
Force civil engineer community. Through technical information, guidance 
and standardized methodology, the directorate enables civil engineers 
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worldwide to execute their expeditionary combat support and emergency 
services missions safely, effectively and efficiently. The directorate has 
five divisions: Explosive Ordnance Disposal; Emergency Management; 

Eire Emergency Services; Expeditionary Engineering; and Requirements 
and Acquisition. 

The AECEC is responsible for building the guidance/direction/regulations Air 
Eorce Civil Engineers will use in an expeditionary environment such as while repairing 
and recovering an airbase after an attack. The AECEC is also responsible for ensuring 
that the Air Eorce Civil Engineer career field is prepared for the future of expeditionary 
warfare by integrating the latest technology into TTPs as well as standard operating 
procedures. 


3. RED HORSE 

Erom AEI 10-209 (Department of the Air Eorce 2012a), the role of a RED 
HORSE unit is described as: 

[The] RED HORSE [unit] directly supports combat air power worldwide. 

They provide air component commanders a dedicated, flexible airfield and 

base heavy construction and repair capability, along with many special 

capabilities that allow the unified [combatant commanders] CCDRs to 

move and support missions as the air order of battle dictates. 

The RED HORSE unit can rapidly repair a damaged airfield, obtained through 
various means. 

4. NAVY SEABEES 

The Navy Seabees are responsible for providing the U.S. Navy with rapid, 
expeditionary construction to include ADR. “Since its inception during the early days of 
World War II, Airfield Damage Repair (ADR) has been one of the Seabees’ core 
competencies” (Pierce 2016). They are required to assess, locate, plot, and repair damage 
done to an airfield in order to enable the rapid use of airpower in a wartime environment. 
The Navy Seabees provide a similar capability to the U.S. Navy that the RED HORSE 
provides to the U.S. Air Eorce. 

The U.S. faces potential threats around the globe in which Seabees may be called 
upon once again to provide ADR services for our nation’s and for our allies’ 
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expeditionary forces. For this reason there has been a renewed focus on their ADR 
capabilities (Pierce 2016). 
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II. 


PROJECT APPROACH 


A. INTRODUCTION 

This chapter outlines the methodology that was utilized when conducting the 
lADAS concept development. All work during this phase of the project fell within the 
Material Solution Analysis Phase shown in Figure 2. 
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Figure 2. DOD Acquisition Life Cycle. Source: Defense Acquisition 

University (2017). 


The following activities are performed to support the development of alternative 
system architectures: 

1. Requirements Analysis. Stakeholder requirements were adapted into system 
requirements that characterized the attributes and performance of the lADAS. 

2. Functional Analysis. lADAS high level functions identified through 
requirement analysis were subsequently decomposed. 
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3. Design (Physical) Synthesis. Physical resources required for all functions 
identified in the functional architecture were detailed. 

4. Alternate lADAS Candidate Architectures. The specific physical resources 
used for each of the two alternate systems were selected using a 
morphological box rooted in the functional analysis and physical synthesis. 

The SE process shown in Figure 3 was used to support the completion of the 
requirements analysis, functional analysis, and design synthesis. Upon completion of 
development of the alternate lADAS systems, an AoA was performed. The AoA was 
performed based on estimated system cost and performance to provide stakeholders with 
sufficient information to understand the benefits, risks, and costs associated with each 
alternative system. 
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Figure 3. Systems Engineering Process. Source: Defense Acquisition 

University (2001). 

The SE technical approach, key SE strategies, programmatic constraints, and 
programmatic assumptions are shown in Table 2. 
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Table 2. lADAS Project Approach, Key Strategies, Constraints, and 

Assumptions 


_ Approach _ 

1. Documented the extent of the ADA problem 

2. Identified the stakeholders and their requirements 

3. Described and evaluated the current overall system of ADA (including 
functional and system architectures) 

4. Described and evaluated the current means to identify UXO 

5. Defined the measures (Measures of Effectiveness (MOEs), Measures of 
Performance (MOPs)) for ADA 

6. Generated and evaluated two alternatives to the current system 

7. Documented the evaluation approach (including the use of Modeling and 
Simulation (M&S) 

8. Developed functional and physical architectures for the alternatives 

9. Prepared an AoA between the current process and the two alternatives _ 

_ Key Strategies _ 

1. Tailored SE process and products using Spec Innovations Innoslate 

2. Deliverables were coordinated with the team through the team leader 

3. Research questions were generated by the team 

4. Data collected through the use of online resources and libraries 

5. M&S were run to estimate how effective the alternative systems operate and 
leveraged tools such as ExtendSim 

6. Cost estimation was performed to the best extent possible using the Center 
for Systems and Software Engineering Constructive Systems Engineering 
Cost Model (COSYSMO)/Constructive Cost Model II (COCOMO II) 

7. Two alternatives were generated 

8. Made sure to keep the functional focused on the “what” to be accomplished, 

and the physical on the “how” things were accomplished _ 

_ Constraints _ 

1. The team had limited access to actual stakeholders 

2. The project was completed within three semesters according to Naval 
Postgraduate School (NPS) guidelines 

3. The project team was fixed to the identified six people and did not expand 
nor contract to complete the project work 

4. Certain physical components examined were not able to meet the 

requirements of the system _ 

Continued on next page. 
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Table 2. 


Continued from previous page. 


_ Assumptions _ 

1. There were enough data for the project through web search and interview 
data collection 

2. Attributes from equivalent components were used for modeling 

3. No real budget existed. There were no expenditures to complete the project. 
There was no procurement 

4. The proposed solution to the problem was not actually implemented 

5. There was no materiel solution to deploy 

6. Not all physical component attributes were readily available to the team. 

Analogies to other, related technologies, were substituted for the purpose of 
this report _ 


B. REQUIREMENTS ANALYSIS METHODOLOGY 

Requirements analysis was performed using the Institute of Electrical and 
Electronics Engineers (IEEE) SE standard PI220 titled '"Standard for System 
Engineering."' Eifteen task areas were taken into consideration during this analysis 
(Schmidt 2002). Taken into account during the requirements analysis were inputs, 
controls, and enablers. Eor the purpose of the lADAS effort, inputs, controls, and 
enablers were leveraged as much as possible to conduct a thorough requirements 
analysis. 

C. FUNCTIONAL ANALYSIS METHODOLOGY 

Functional analysis is an SE process activity that transforms the requirements into 
functions necessary for the system to accomplish its mission. The system functions then 
guide the system design. The objective at this stage was to identify what the system will 
accomplish and not how the system will do it. “The purpose of the ‘functional analysis’ is 
to present an overall integrated and composite description of the systems’ functional 
architecture, to establish a functional baseline for all subsequent design and support 
activities, and to provide a foundation from which all physical resource requirements are 
identified and justified; that is, the system’s physical architecture” (Blanchard and 
Fabrycky 2011). Enhanced functional flow block diagrams were used to describe the 
functional architecture. 
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D. PHYSICAL ARCHITECTURE METHODOLOGY 


The physical architecture of the lADAS system identified the physical resources 
required for all functions identified in the functional architecture (Blanchard and 
Fabrycky 2011). The tool leveraged for the physical architecture was the morphological 
box. Research was performed in order to determine the physical components that had the 
capability to deliver the appropriate functionality to address the needs of lADAS. A 
comparison of options resulted in a wide variety of potential systems that needed to be 
reduced further to get down to the target of evaluating two system concepts. The SE tool 
used to perform this comparison was the Pugh matrix. The purpose of the Pugh matrix 
was to take the multitude of requirements and begin to reduce the alternatives to a 
manageable number. Once the Pugh matrix analysis was complete, the two lADAS 
alternatives were identified. 

E. ANALYSIS OF ALTERNATIVES METHODOLOGY 

The SE process built the foundation necessary for an AoA. The requirements had 
been decomposed from the stakeholders. Several representations of the functional 
components were documented so the system could be understood from a visual 
perspective. This, in turn, was further elaborated upon by creating the physical 
architecture of the I AD AS alternatives. The main components of the AoA were cost and 
effectiveness. Eirst, the cost per component was researched, either through 
documentation from existing systems or relative costs from components that delivered 
similar functionality. Second, the performance data was derived from similar components 
available at the time of the study. All of the data was collected, summarized and used as 
inputs to the system model. Modeling is described in the next chapter. 

F. MODELING METHODOLOGY 

I. Cost Modeling Methodology 

A cost analysis was performed to provide the estimated ownership cost, 
associated for each of the three systems, through a 10-year life cycle. This approach 
captured the cost for a single implementation of each alternative over the defined 10-year 
life cycle and included cost incurred for: 
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• research and development (R&D) (lADAS I and II only) 

• the SE (lADAS I and II only) 

• personnel 

• operations and support (O&S) 

The parametric method for estimating was chosen because there was limited 
program and technical definition. The NFS System Cost Model Suite software was used 
to develop estimated systems engineering, software, and hardware costs over a 10-year 
life cycle for each of the alternative systems. The comparison was made between all three 
systems, comparing their ownership costs against each other, and determining the best 
value against the time to accomplish the mission of ADA and reporting. 

2. Effectiveness Modeling Methodology 

The effectiveness methodology (Figure 4) was the sum of the processes used to 
conduct the effectiveness assessment (EA). 

This methodology was designed to compare the effectiveness of the three ADAS 
systems based on their military and operational worth. The EA addressed both 
operational effectiveness and operational suitability. The lADAS mission tasks were 
developed based on the requirement analysis performed previously. The MOE estimates 
were developed to assess the ability of the alternative lADAS systems to satisfy the 
developed mission tasks. The MOEs are a measure of how well a mission task was 
accomplished through using a given alternative system. 
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Figure 4. Effectiveness Assessment Methodology. Source: Air Force 

Material Command (2010). 


The effectiveness analysis was conducted at the engagement level to model the 
interaction between lADAS alternatives versus a single threat situation, as shown in 
Figure 5. This level of analysis was chosen due to the time constraints associated with the 
project and the increased complexity of analyzing at a higher level especially with respect 
to M&S. 

The methods used to support the lADAS effectiveness assessment included 
modeling and simulation, comparative analysis with legacy systems, and engineering 
assessments. 
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Figure 5. Effectiveness Analysis Methodology. Source: Air Force Material 

Command (2010). 


G. DESIGN REFERENCE MISSION 

The ADA is most critical after an enemy attack when seconds matter in returning 
the airfield back to operational status. Because the purpose of this project was to focus on 
the ADA, the DRM was defined to be beginning after an enemy attack on a U.S. airfield. 
In order to design a solution that is operationally feasible for a majority of the airfields, it 
is understood that the ADA should be all encompassing to include: 

1. Various types of airfield surfaces such as paved, unimproved, or dirt 

2. Airfields of various dimensions to include length and width 

3. The UXO types, size, and quantity 

4. Weather conditions such as hot, cold, rain, wind, fog, or snow 
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5. Available lighting such as day, night, or twilight. 


A DRM that is limited in scope was generated due to the limited resources 
available to include the time to complete the study and number of personnel conducting 
the study. Interested stakeholders can expand the results of this study to determine if the 
recommended solutions also work well with the other environmental considerations listed 
above. 

Development of the DRM involved researching the characteristics of various 
foreign bombs likely to be involved in an airbase attack in order to have a realistic input 
into the model. After careful consideration, the Russian RBK-500 BetAB Cluster Bomb 
(see Figure C-1) was selected as one of the weapons to be used against an airfield in the 
theater of interest. The Russian RBK-500 BetAB carries 12 BetAB, which are concrete 
penetrating bomblets for anti-airfield attacks (Jane's Air Launched Weapons 2007). It is 
reported that each bomblet (BetAB) is capable of penetrating 400 mm concrete and 
causes a damage area of 4 m^ (Jane's Air Launched Weapons 2007). The BLU-97 was 
selected as a comparable weapon to determine the failure rate of the RBK-500: “The 
official failure rate of the BLU-97 is seven percent, but failure rates of at least 16% have 
been observed in Afghanistan. The failure rate of the Soviet sub-munitions is not known. 
Their fuses would be less sensitive, making it more difficult to unintentionally detonate a 
failed sub-munition” (Handicap International 2007). Based on this similarity, a dud rate 
of 16% was selected for the DRM. The RBK-250 AO-1 was used for comparison 
purposes to determine the maximum footprint for the RBK-500 BetAB. The RBK-250 
AO-1 is equipped with 150 fragmentation bomblets. The canister is 2120 mm long, 
325 mm in diameter, and weighs 273 kg, including 150 kg of sub-munitions. The 
maximum footprint area is 4,800 m^ (The Fighter Collection & Eagle Dynamics, Inc. 
2013). The RBK-250 AO-1 is an antipersonnel CBU. The similarity between the two 
weapons allows for the “maximum footprint” to be the used for the DRM. 

An RBK-500 Cluster Bomb Unit (CBU) filled with A02.5RTs is also part of the 
DRM. The main reason for this is to add UXOs to the model. The AO-2.5RT is an anti¬ 
personnel fragmentation sub-munition, which could be used to slow down the damage 
assessment and repair phases. It carries 108 AO-2.5RT sub munitions (fragmentations) 
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(International Campaign to Ban Landmines 2012). It also has a footprint of 224,000 ft^ 
(267 ft. radius) (SU-27 Flanker 2017). Based on the same rationale provided for the 
RBK-500, a dud rate of 16% was used for the DRM. 

After careful consideration, the Russian FAB-500 T General Purpose Bomb (see 
Figure C-3) was selected as the other weapon to be used against an airfield in the theater 
of interest. In the case of slow blasting crater, the FAB-500 T had a depth 13 m, diameter 
of 22.5 m, and the radius of the separation of fragments was 430 m. Such an effect is 
possible due to the fact that the bomb at high speed penetrates and digs into the ground 
and then later explodes (Global Security.org 2016). The DRM weapon characteristics are 
summarized in Table 3. 


Table 3. DRM Weapon Information 


Weapon 

Number of 
Sub¬ 
munitions 

Dud 

Rate 

(%) 

Crater Size/ 
Damage 
Footprint 

Crater 

Depth 

Clean up/ 
Spalling 
Diameter 

Max 

Dispersal 

Russian 

RBK-500 BetAB 
Cluster Bomb 

12 BetAB 

16 

4 m^ 

N/A 

N/A 

40 m/ 
(-130 ft) 

Russian 

RBK-500RTM 

AO Cluster 

Bomb 

108 AO- 
2.5RT 

16 

20810.3 m2 
(224,000 ft2) 

N/A 

N/A 

81.4 m 
(267 ft) 

Russian FAB-500 
T Bomb 

N/A 

N/A 

398 m2/ 
(4273 ft 2) 

13 mJ 
(43 ft) 

430 m/ 
(1411 ft) 

N/A 
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The DRM scenario follows: 


The USAF has established Chinoski Air Base (AB), a forward operating airbase 
in fictional Kasnia, which was considered a hostile area. Chinoski AB has two primary 
runways of concern and the runways are the focus of the ADA. These runways run 
parallel to each other and are 5676 and 6521 feet in length and 150 feet wide. Chinoski 
AB was prepared for an attack, and they prepositioned and dispersed equipment and 
personnel as a preemptive measure as well as established and assigned members for the 
ADA and UXO team in addition to the EOC. A grid map, a map with X, Y coordinates 
for reference, of the airfield was created and dispersed to the personnel within the EOC 
and the ADAT to help support the determination of where the damage and UXO 
locations were specified. At 1000 local time, the base RADAR detected enemy aircraft 
approaching Chinoski AB, and the base commander determined there was not enough 
time to launch a counter attack. The base was immediately put into an “Alarm Red” 
condition, which indicated that all personnel needed to take immediate cover in 
preparation of an attack on Chinoski AB. It should be noted that each base follows an 
overarching protocol such as those described in AFP AM 10-219 (Department of the Air 
Eorce 2008) for Alarm Red conditions, with each individual base establishing local 
procedures. At 1030, the attack concluded, the base was put into “Alarm Black,” and 
ADA started. Table 4 provides the detailed information about the attack including the 
weapon used, the grid location where it landed, and the damage produced. Eigure 6 
shows a layout of the takeoff and landing surfaces as well as the damage as a result of the 
attack on Chinoski AB. Additional details can be found in Appendix C, Tables C-1 
through C-3 . 
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Table 4. Detailed Attack Information 


Weapon 

Location 
Dropped (X, Y 
coordinates) 

Damage 

RBK-500 

BetAB (1) 

3261,700 

• 12 BetAB bomblets landed on runway 27R/9L 

• 4 Bomblets did not function and are currently 
classified as a UXO 

• 0 Bomblets landed on runway 9R/27L 

RBK-500 

BetAB (2) 

1000,75 

• 0 Bomblets landed on runway 27R/9L 

• 12 BetAB bomblets landed on runway 9R/27L 

• 5 Bomblet did not function and are currently 
classified as a UXO 

RBK-500 AO- 
2.5RT 

500,700 

• 43 Bomblets landed on runway 27R/9L 

• 4 Bomblets did not function and are currently 
classified as a UXO 

FAB-500T(1) 

2130,700 

• Landed on runway 27R/9L 

• Formed a crater 22.6 m (74 ft) wide and 13 m 
(43 ft) deep, with debris and spalling diameter of 
430 m (1411 ft) 

FAB-500 T (2) 

5000,75 

• Landed on runway 9R/27L 

• Formed a crater 22.6 m (74 ft) wide and 13 m 
(43 ft) deep, with debris and spalling diameter of 
430 m (1411 ft) 

FAB-500 T (3) 

4000,700 

• Landed on runway 27R/9L 

• Formed a crater 22.6 m (74 ft) wide and 13 m 
(43 ft) deep, with debris and spalling diameter of 
430 m (1411 ft) 

FAB-500 T (4) 

3261,75 

• Landed on runway 9R/27L 

• Formed a crater 22.6 m (74 ft) wide and 13 m 
(43 ft) deep, with debris and spalling diameter of 
430 m (1411 ft) 
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This DRM is based on a forward operating airbase in a hostile area. Based on enemy 
aircraft approaching, the base was put into an “Alarm Red” condition. Following the 
attack, the base was put into an “Alarm Black” condition. The damage from the enemy 
attack is noted on the figure. 

Figure 6. DRM. Adapted from Aircraft Owners and Pilots 

Association (2017). 
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III. SYSTEMS DESCRIPTIONS 

A. INTRODUCTION 

During this phase of the project, the system requirements were refined based on 
stakeholder needs. The functional and physical architectures were developed. The current 
system was described and the two alternate systems created. 

B. REQUIREMENTS ANAUYSIS 

The stakeholder needs (Table 5) for the new lADAS were identified based on the 
user and problem statement, literature research, and the stakeholder analysis. The data 
flow diagram (DFD) (Figure 7) was developed based on the stakeholder needs to 
determine the system boundary, system inputs, system outputs, and internal/external data 
flows. 


Tables. Stakeholder Needs 


Reference 

Description 

I.O 

Damage Assessment 

1.1 

lADAS shall detect when an airfield has been damaged. 

1.2 

lADAS shall classify the type of airfield damage. 

1.3 

lADAS shall locate and measure airfield damage. 

2.0 

UXO Assessment 

2.1 

I AD AS shall detect UXO on airfield surfaces. 

2.2 

lADAS shall classify the type of UXO on airfield surfaces. 

2.3 

lADAS shall provide a location of UXO on airfield surfaces. 

2.4 

lADAS shall automatically transmit damage and UXO data into 
GeoExPT. 

2.5 

lADAS shall not expose personnel to explosive hazards during UXO 
and damage assessment activities. 

3.0 

Time Assessment 

3.1 

lADAS shall complete damage and UXO assessment and reporting in 
less than 30 minutes (objective)/45 minutes (threshold). 
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The DFD is comprised of the ADAS, the ADR team, external sensor systems, and the 
damaged area. 


Figure 7. lADAS DFD 


The results from examining the 15 task areas during requirements analysis are 
provided in Table 6. From here, the functional (what the system should do) vs. non¬ 
functional requirements (how well the system works) for lADAS were developed. 
Table 7 provides the Functional Requirements Traceability Matrix and Table 8 provides 
the Non-Functional Requirements. 
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Table 6. Requirements Analysis Results 


Task 

Results 

Scope Limitation 

Customer 

expectations 

At the highest level, the expectation of the 
lADAS was to reduce the time for detailed 
assessment and quantity of AD AT required to 
complete an ADA. Information acquired from 
the interview was used in this report. 


Project and 

enterprise 

constraints 

Constraints imposed upon the system were 
primarily the policies and procedures that were 
required to perform an ADA successfully. Other 
constraints were the lack of financial and human 
resource allocations to the effort. 


External 

constraints 

One of the primary hurdles the system 
development faced was the level of 
advancement in the current technology base. 


Interfaces 

The interfaces used in lADAS provide 
communication between hardware components 
to ensure the system was able to complete the 
mission. Software was used to coordinate 
sensors and communicate appropriate messages 
between lADAS sensors and also to the User. 

Due to time 
constraints, the 
primary interface 
was limited to the 
GeoExPT. 

Utilization 

Environments 

The environment in which the lADAS will 
operate will vary from location to location, all 
over the world. Weather conditions of all types 
may be encountered, including but not limited 
to, rain, snow sun, wind, ice, dust, and fog. 
Extreme temperature ranges and relative 
humidity may be encountered. All land based 
topologies were taken into account for the 
lADAS system. Topologies include but were not 
limited to mountain, desert, plains, and 
vegetation. 

Due to time 
constraints, the 
environment under 
consideration was 
limited to daylight 
and fair weather 
conditions. 


Continued next page. 
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Table 6. 


Continued from previous page. 


Task 

Results 

Scope Limitation 

Life cycle 

process 

concepts 

The lADAS will follow the DOD acquisition life 
cycle, as specified in DODI 5000.02, with all 
relevant key life cycle process milestones. 

Driving down cost and risk over the full life 
cycle of the system were the key drivers for 
following this process (Under Secretary of 
Defense (AT&L) 2003). Regular reviews 
throughout the life cycle ensured that the lADAS 
baseline was developed to be producible, 
testable, operable, supportable, and trainable. 
Reviews include Initial Technical Review, 
Alternative System Review, System 

Requirements Review, Integrated Baseline 
Review, System Functional Review, Preliminary 
Design Review, Critical Design Review, Test 
Readiness Review, Flight Readiness Review, 
System Verification Review, Functional 
Configuration Audit, Production Readiness 
Review, Operational Test Readiness Review, 
Physical Configuration Audit, and In-Service 
Review. 

These reviews were 
outside the scope of 
this report. 

Functional 

Requirements 

The functions of the lADAS were the 
foundational elements that enabled the system to 
accomplish its mission. The basic high level 
functions of the lADAS were to detect, identify, 
assess, classifv, measure, and report. See Table 7 
for a listing of the functional requirements. 


Operational 

scenarios 

The operational scenario for the lADAS was 
defined in the DRM. The lADAS must be 
capable of performing a detailed analysis of a 
damaged airfield consisting of runways, 
taxiways, and aprons with UXO present. A 
standard dual runway is nominally sized 12,000 
feet by 150 feet each and ramps, aprons and 
taxiways nominally total 86,000 by 100 feet. 


MOE and 
Measure of 
Suitability 
(MOS) 

The performance measures used to determine the 
ability of the lADAS to meet the customer’s 
mission mainly focus on effectiveness. 

MOEl: Damaged Assessed 

MOE2: UXO Assessed 

MOST Safety 

MOS2: Operability 

MOS3: Reliability 

Due to time 
constraints, the 

MOS evaluation 
was outside the 
scope of this report. 


Continued next page. 
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Table 6. 


Continued from previous page. 


Task 

Results 

Scope Limitation 

System 

boundaries 

Physical and software components developed 
specifically for use in the lADAS were under 
design control of this activity. Any components 
leveraged from outside systems, including 
government furnished equipment was not under 
this system boundary and was the responsibility 
of the owning Program Manager. Interfaces 
between the lADAS and external systems, to 
include both hardware and software interfaces, 
were under the lADAS system boundary. 


Performance 

Requirements 

Assessing the damage and UXO 
evaluation became the basis for the 
performance requirements of the lADAS 
systems. 


Modes of 
operation 

The lADAS must have multiple modes of 
operation (manual, autonomous, or a 
combination of both). 


Technical 

performance 

measures 

(TPMs) 

The key indicators of system performance for 
the lADAS were related to the most important 
performance parameters derived from the 
system requirements. The survey time and the 
accuracy of the reporting the damage location 
and size were of the utmost importance and were 
defined as the TPM for the lADAS. 


Physical 

characteristics 

The physical characteristics of lADAS were 
appropriate to satisfy all given environmental 
requirements. 

Due to time 
constraints, a 
physical 
characteristics 
evaluation was 
outside the scope of 
this report. 

Human 

Systems 

Integration 

(HSI) 

HSI aspects of the lADAS were designed to 
standards as specified in the MIL-STD-I472G 
DOD Design Criteria Standard for Human 
Engineering (Department of Defense 2012). 

Due to time 
constraints, HSI 
evaluation was 
outside the scope of 
this report. 
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Table 7. Functional Requirements Traceability Matrix 


lADAS 

Specification 

Reference 

(From 

Innoslate) 

lADAS System Specification Requirement Description^ 

From 

Stakeholder 

Needs 

1.1.1 

lADAS shall detect craters on paved surfaces greater than or 
equal to 90 percent of the total number of craters. 

1.1 and 1.2 

1.1.2 

lADAS shall detect craters on airfield infield surfaces greater 
than or equal to 50 percent of the total number of craters. 

1.1 and 1.2 

1.1.3 

lADAS shall detect craters on semi-prepared airfield surfaces 
greater than or equal to 80 percent of the total number of 
craters.^ 

1.1 and 1.2 

1.2.1 

lADAS shall detect camouflets on paved surfaces greater 
than or equal to 80 percent of the total number of camouflets. 

1.1 and 1.2 

1.2.2 

lADAS shall detect camouflets on paved airfield infield 
surfaces greater than or equal to 50 percent of the total 

'y 

number of camouflets. 

1.1 and 1.2 

1.3.1 

lADAS shall detect UXOs on Paved Surfaces greater than or 
equal to 80 percent of the total number of UXOs. Objective 
is 90 percent of the total number of UXOs. 

2.1 

1.3.2 

lADAS shall detect UXOs on semi-prepared airfield surfaces 
greater than or equal to 80 percent of the total number of 

UXOs. Objective is 90 percent of the total number of UXOs. 

2 

2.1 

1.3.3 

lADAS shall detect UXOs on airfield infield surfaces greater 
than or equal to 50 percent of the total number of UXOs. 

'y 

Objective is 75 percent of the total number of UXOs. 

2.1 

1.4 

lADAS shall classify UXO by major class category lAW the 
Airman’s Manual: Class A (Large Bombs); Class B 
(Rockets and Missiles); Class C (Projectiles and Mortars); 
Class D (Landmines); Class E (Bomblets); and Class F 
(Rocket Propelled Grenades and Grenades).^ 

2.2 

1.5.1 

lADAS shall measure apparent diameter of surface damage 
for camouflets, craters and spalls within 20 percent of actual 
diameter. 

1.3 

1.5.2 

lADAS shall measure apparent damage of surface damage 
for camouflets, craters and spalls within 10 percent of actual 
diameter. 

1.3 

1.5.3 

lADAS shall measure apparent diameter of a camouflets 
under a surface within 10 percent of actual diameter. 

1.3 

1.6.1 

lADAS shall locate damage by the center point of the object. 

1.3 

1.6.2 

lADAS shall report horizontal positional accuracy of less 
than or equal to ten feet. Objective is a horizontal positional 
accuracy of two (2) feet. 

1.3 


Continued next page. 
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Table 7. _ Continued from previous page. 


lADAS 

Specification 

Reference 

(From 

Innoslate) 

lADAS System Specification Requirement Description^ 

From 

Stakeholder 

Needs 

1.6.3 

lADAS shall report location data in Mapping Grade 
precision using the Military Grid Reference System (MGRS). 

2.4 

1.9.1 

lADAS shall complete damage and UXO assessment and 
reporting in less than 30 minutes. 

3.1 

NOTES: 

'Derived from the RFI in support of planning/acquisition strategy development for 
RADAS dated 07 June 2012 (Department of the Air Force 2012b). 

^Due to time constraints, the requirements identified were not evaluated as part of the 
AoA. 

^Technology to identify and classify UXO is still relatively new. The bigger push at 
this time is to identify damage. No known objective or threshold values for correctly 
classifying the UXO are available at this time. 


Table 8. Non-functional Requirements 


lADAS 

Specification 

Reference 

(From 

Innoslate) 

LADAS System Specification Requirement 
Description^ 

From 

Stakeholder 

Needs 

1.7.1 

lADAS shall be assessable in day, night^ and civil 
twilighC lighting conditions. 

4.2 

1.7.2 

lADAS shall operate in temperature conditions 
from -45 to -1-130 degrees Fahrenheit^. 

4.2 

1.7.3 

lADAS shall be able to assess through obscurants 
such as fog, fog oil, and brass flakes. ^ 

4.2 

1.7.4 

lADAS shall operate in relative humidity up to and 
including 100 percent. ^ 

4.2 

1.7.5 

lADAS shall perform assessment through 
precipitation and accumulation of rain 0.3’7hour, 
snow accumulation of 3’7hour, and dry and wet 
surfaces with water puddles.^ 

4.2 

1.7.6 

lADAS shall be operable from a sheltered location 
near the airfield. 

4.3 

1.7.7 

lADAS shall be remotely operable from an 
installation operations center. 

4.3 


Continued next page. 
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Table 8. _ Continued from previous page. 


lADAS 

Specification 

Reference 

(From 

Innoslate) 

LADAS System Specification Requirement 
Description^ 

From 

Stakeholder 

Needs 

1.7.8 

lADAS shall be small-arms resistant and blast 
resistant. ^ 

4.2 

1.7.9 

lADAS shall be eapable of withstanding Nuelear, 
Biologieal, and Chemieal (NBC) deeontamination 
proeedures. ^ 

4.2 

1.8.1 

lADAS shall be eapable of being operated and 
maintained (exeluding depot-level maintenanee) 
while wearing NBC mission oriented proteetive 
posture (MOPP)-IV elothing for indoor and outdoor 
tasks. ^ 

4.1 

1.8.2 

lADAS shall be eapable of being operated and 
maintained while wearing eold weather elothing for 
outdoor tasks or tasks performed in non¬ 
temperature eontrolled shelters. ^ 

4.1 

NOTES: 

'Derived from the REI in support of planning/aequisition strategy development for 
RADAS dated 07 June 2012 (Department of the Air Eoree 2012b). 

^Due to time eonstraints, the requirements identified were not evaluated as part of the 
AoA. 

^Due to time eonstraints, temperatures outside ambient were not evaluated as part of 
the AoA. 


The final step for the requirements analysis was to develop the MOEs based on 
the requirements (see Table 9). Three MOEs were utilized to evaluate the effeetiveness of 
the systems. 


Table 9. lADAS MOEs 


MOE 1 

Damage Assessment Time 

MOE2 

Percent Damage Assessed 

MOES 

Percent UXO Assessed 
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C. CURRENT ADAS SYSTEM AND FUTURE lADAS CONCEPTS 

The next step in the process of determining the potential alternatives for the 
lADAS system was to determine the components used in the current ADAS system, and 
those potential components which could be leveraged for the lADAS system. The tools 
used to perform those activities were the morphological box and the Pugh matrix SE 
concepts. 

Starting with the morphological box concept, the functional architecture was 
referenced in order to create the categories of physical components. The next step was to 
research the broad spectrum of tools which could deliver the required capabilities within 
each of those categories. 

The morphological box for the current ADAS system was fairly straightforward, 
since the process was mainly a manual effort performed by the ADAT resources. Table 
10 shows both the functional categories (columns) and physical components (rows) 
which make up the concept used today. The column entitled Mechanism refers to the 
mode of transportation used to convey the ADAT team to the area of interest which is 
ground travel via a High Mobility Multipurpose Wheeled Vehicle (HMMWV) tactical 
vehicle. The column entitled Sensor refers to the components used to gather the necessary 
data for the assessment of airfield damage, UXO identification, and location information. 
The third column entitled Data Processor indicates the resource utilized to put the ADA 
report information together. The final column entitled Communication refers to the 
hardware utilized to communicate the report data back to the EOC. 


Table 10. Current ADAS System Components 


Mechanism 

Sensor 

Data Processor 

Communication 

HMMWV 

Eyes 

Human 

Radio 

Tape Measure 


The next step was to take the morphological box categories and expand them to 
cover the “art of the possible” as it applied to future concepts to deliver the same 
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capability as the manual process applied by the ADAT resources. The results of that 
effort are documented in Table 11. 


Table 11. The Future lADAS System Components 


Mechanism 

Sensor 

Data Processor 

Communication 

RPA 

LiDARi 

CPU2 

Hardwire 

Stationary Setting 

RADAR 


Removable Storage 

UGV^ 

Acoustic Imaging 


Wireless 

Satellite 

Infrared Imaging 




Day Camera 




Imbedded Sensors 



^ Light Detection and Ranging ^Central Processing Unit 

^ Unmanned Ground Vehicle 


A design feature interaction analysis was used where all of the components were 
evaluated against each other. The purpose was to sort out any combinations which would 
not be compatible with one another. Using this method, the complete set of 72 options 
was reduced to 50 options. These options were further reduced based on USAF studies in 
which UGV or Acoustics solutions were eliminated due to being functionally improbable. 
Additionally, based on USAF studies, satellite and imbedded sensors were eliminated due 
to cost. This reduced the number of viable options to 20. The results are captured in 
Table 12. 


Table 12. Final List of Options for Pugh Matrix Analysis 


Option 

Mechanism 

Sensor 

Data Processor 

Communication 

1 

RPA 

LiDAR 

CPU 

Wireless 

2 

RPA 

RADAR 

CPU 

Wireless 

3 

RPA 

Infrared Imaging 

CPU 

Wireless 

4 

RPA 

Day Camera 

CPU 

Wireless 

5 

RPA 

LiDAR 

CPU 

Removable Storage 

6 

RPA 

RADAR 

CPU 

Removable Storage 

7 

RPA 

Infrared Imaging 

CPU 

Removable Storage 

8 

RPA 

Day Camera 

CPU 

Removable Storage 


Continued next page 
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Table 12. _ Continued from previous page 


Option 

Mechanism 

Sensor 

Data Processor 

Communication 

9 

Stationary 

Tower 

LiDAR 

CPU 

Hardwire 

10 

Stationary 

Tower 

RADAR 

CPU 

Hardwire 

11 

Stationary 

Tower 

Infrared Imaging 

CPU 

Hardwire 

12 

Stationary 

Tower 

Day Camera 

CPU 

Hardwire 

13 

Stationary 

Tower 

LiDAR 

CPU 

Wireless 

14 

Stationary 

Tower 

RADAR 

CPU 

Wireless 

15 

Stationary 

Tower 

Infrared Imaging 

CPU 

Wireless 

16 

Stationary 

Tower 

Day Camera 

CPU 

Wireless 

17 

Stationary 

Tower 

LiDAR 

CPU 

Removable Storage 

18 

Stationary 

Tower 

RADAR 

CPU 

Removable Storage 

19 

Stationary 

Tower 

Infrared Imaging 

CPU 

Removable Storage 

20 

Stationary 

Tower 

Day Camera 

CPU 

Removable Storage 


The next step was to start to build the options using the Pugh matrix. The 
requirements were reduced down to a manageable amount so as to reduce the complexity 
of the calculations. The major requirements for the lADAS system were considered: 
damage location, damage size, damage accuracy, UXO location, UXO identification, 
reporting, and lADAS portability. Additional non-functional requirements considered 
included affordability, maintainability, reliability, and survivability. For each option a 
entered into each cell to represent if the new concept is significantly 
better “-t,” worse or the same “S” as the datum concept. The utilization of the Pugh 
matrix helped to quantitatively analyze the various combinations of system components 
being brought together to create various design alternatives. By going through the Pugh 
matrix process, the lADAS alternatives were compared based on how they addressed the 
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requirements from a complete system perspective (see Table 13). Table 14 provides the 
results of the Pugh matrix analysis. 
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Table 13. Pugh Matrix Analysis 


Requirement 

Number 

Concept Critical to 
Satisfaction or 
Requirement 

Importance 

Rating 

0 

IRLW 

2RRW 

3RIW 

4RDW 

5RLR 

6RRR 

7RIR 

8RDR 

9SLH 

lOSRH 

11 SIH 

12 SDH 

13SLW 

14SRW 

15 SIW 

16SDW 

17 SLR 

18SRR 

19 SIR 

20 SDR 

1.1.1 

Damage count 
accuracy for craters 
on paved surfaces 

10 

DATUM 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

1.2.1 

Damage count 
accuracy for 
camouflets on paved 
surfaces 

9 


+ 


+ 

+ 



+ 

+ 



+ 

+ 



+ 

+ 



+ 

1.3.1 

UXO detection 
accuracy on paved 
surfaces 

9 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

1.4 

UXO classification 

6 

- 

" 

- 

+ 

+ 

- 

” 

- 

+ 

- 

" 

- 

+ 

+ 

- 

” 

- 

+ 

+ 

- 

" 

- 

+ 

1.5.1 

Surface damage 
measurement 
accuracy for craters 

5 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 













1.5.1 

Surface damage 
measurement 
accuracy for 
camouflets 

2 

S 

+ 

S 

s 

S 

+ 

S 

S 

s 

+ 

s 

s 

s 

+ 

s 

s 

s 

+ 

s 

s 

1.5.1 

Surface damage 
measurement 
accuracy for spalls 

6 

s 

S 

s 

s 

s 

S 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

s 

1.5.3 

Subsurface damage 
measurement 
accuracy for 
camouflets 

7 

s 

+ 

s 

s 

s 

+ 

s 

s 

s 

+ 

s 

s 

s 

+ 

s 

s 

s 

+ 

s 

s 

1.6.1 

Damage location at 
center point of 
object 

4 

s 

s 

s 

s 

s 

s 

s 

s 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

1.6.2 

Damage location 
accuracy 

6 









+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

1.6.3 

Damage location 
using MGRS 

3 

s 

s 

s 

s 

s 

s 

s 

s 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 


Continued next page. 
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Table 13. 


Continued from previous page. 


Requirement 

Number 

Concept Critical to 
Satisfaction or 
Requirement 

Importance 

Rating 

0 

IRLW 

2RRW 

3RIW 

4RDW 

5RLR 

6RRR 

7RIR 

8RDR 

9SLH 

lOSRH 

11 SIH 

12 SDH 

13SLW 

14SRW 

15 SIW 

16SDW 

17 SLR 

18SRR 

19 SIR 

20 SDR 

1.9.1 

Damage and UXO 
assessment in timely 
fashion 

10 


+ 

+ 

+ 

+ 





+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 





1.7.1 

lADAS accessible 
during the day 

4 

S 

S 

- 

+ 

S 

S 

- 

+ 

S 

S 

- 

+ 

S 

S 

- 

+ 

S 

S 

- 

+ 

1.7.6 

lADAS co-located 
with the airfield 
being assessed 

7 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

1.7.7 

lADAS remotely 
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+ 

+ 

+ 
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+ 
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Table 14. Pugh Matrix Results 



1 RLW 

2RRW 

3RIW 

1 

1 4 RDW 

5RLR 

6RRR 

7RIR 

8RDR 

9SLH 

10 SRH 

11 SIH 

12 SDH 

13 SLW 

14SRW 

15 SIW 

16 SDW 

17 SLR 

18 SRR 

19 SIR 

20 SDR 

Sum of Positives 

8 

11 

9 

13 

8 

10 

8 

11 

8 

9 

8 

12 

8 

9 

7 

13 

7 

8 

7 

9 

Sum of Negatives 

4 

4 

4 

1 

4 

6 

6 

3 

6 

8 

7 

3 

6 

8 

7 

2 

8 

10 

8 

5 

Sum of Neutrals 

7 

5 

6 

6 

8 

5 

6 

6 

7 

5 

6 

7 

7 

5 

6 

6 

7 

5 

6 

7 

Positives - Negatives 

4 

7 

5 

12 

4 

4 

2 

8 

2 

1 

1 

9 

2 

1 

0 

11 

-1 

-2 

-1 

4 

Weighted Sum of Positives 

63 

81 

73 

98 

62 

72 

63 

82 

53 

53 

51 

76 

53 

53 

44 

84 

43 

43 

41 

46 

Weighted Sum of Negatives 

31 

28 

25 

6 

32 

47 

45 

26 

42 

57 

48 

19 

42 

57 

48 

12 

62 

77 

61 

39 

Weighted Sum of Neutrals 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Weighted Positives - 
Weighted Negatives 

32 

53 

48 

92 

30 

25 

18 

56 

11 

-4 

3 

57 

11 

-4 

-4 

72 

-19 

-34 

-20 

7 
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Based on the results, the lADAS I system for further consideration is an RPA 
with a day camera and wireless sensors with a score of 92. The lADAS II system for 
further consideration is a stationary tower with day camera and wireless sensors with a 
score of 72. Both of these systems are highlighted in green in Table 13 and Table 14 . 
Honorable mentions include a stationary tower with a day camera and hard-wired sensors 
with a score of 57 and an RPA with day camera and a removable hard drive with a score 
of 56. These two systems are highlighted in yellow in Table 13 and Table 14 . 

D. SYSTEMS DESCRIPTION SUMMARY 

The next step in the SE process was to define the various artifacts that would 
describe the current ADAS solution. This section includes the functional analysis, 
physical architecture, and CONOPS of the current ADAS, lADAS I, and lADAS II 
systems. 

I. Current ADAS 

a. Functional Analysis 

A functional architecture was generated for the current ADAS system. This top- 
down decomposition showed the functions that were performed for a notional ADA 
mission. Figure 8 provides the top three levels of the current ADAS system. The current 
ADAS system is broken down into Perform Initial Reconnaissance, Perform Detailed 
Damage Assessment, and Compose Damage Assessment Report. Within these functions, 
the ADAS must Observe and Perform a visual inspection, assess airfield damage, and 
assess UXO on airfield. Figure 9 provides further decomposition for the Assess Airfield 
Damage action. Figure 10 provides further decomposition for the Assess UXO on 
Airfield action. 
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Figure 8. ADAS Mission Functional Architecture Hierarchy Chart (First 

Three Levels) 
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Figure 9. Assess Airfield Damage Decomposed 



Figure 10. Assess UXO on Airfield Decomposed 
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The functional architecture was broken out into multiple levels. The first level 
was the mission of the ADAS which was comprised of the assess airfield damage, assess 
UXO, and interoperate with existing airfield damage reporting tools level two functions. 
These level two functions were then decomposed and refined into level three and four 
functions. 

The existing ADAS must assess airfield damage which entailed detecting, 
classifying, measuring, and reporting on the damage. Similarly, the ADAS performed 
these same functions to assess any UXO on the airfield. Interoperability with existing 
airfield damage reporting tools was critical for the rapid reporting and processing of data. 

The EFFBD defines task sequences and their relationships. As can be seen in 
Figure 11, the top level of the ADAS system was comprised of three functions; perform 
initial reconnaissance, perform detailed damage assessment, and compose damage 
assessment report. The functions that the ADAS was performing were decomposed 
further to lower level FFBD, matching the actions in the functional architecture hierarchy 
chart. The actions were traceable through all of the functional levels. 



Figure 11. Current ADAS EFFBD 
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The next step in developing the functional architecture was to decompose all of 
the functions into the Integrated Computer Aided Manufacturing Definition for Function 
Modeling (IDEFO) format. The IDEFO diagram of the current ADAS shows the data 
flow, system control, and overall functional flow. Figure 12 shows the IDEFO diagram of 
the current ADAS. This diagram depicts controls, inputs, outputs, and mechanisms used 
to control each function. 

For the current ADAS, the controls are the Base Alarm, EOC Communication, 
and Raw Assessment Data. Inputs include the visual damage observation data and the 
visual UXO observation data. The mechanisms used in the functions are Observations, 
Trained Personnel, and the ADAT. Outputs include the Compiled Report, Quantity, 
Eocation, Damage Type, and Size information. 



Eigure 12. Current ADAS IDEEO Diagram 


Eollowing preparation of the IDEEO diagram, it was necessary to analyze the 

timeline of events for the current ADAS functional process. The sequence of events 

occurs between the EOC, ADAT, and Tape Measure. The EOC sends the ADAT 
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Deployment Order to the ADAT who then use the information provided by the tape 
measure to gain Raw Assessment Data. The ADAT then relays the Damage Notifieation 
and Measurements, as well as UXO Notification and Measurements to the EOC. The 
ADAT also provides the Compiled Report to the EOC. The sequence diagram can be 
seen in Eigure 13. 
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Eigure 13. Current ADAS Sequence Diagram 


Breaking down the activity further was a class diagram to describe the ADAS 
classes. The classes included attributes, operations, and parameters which were used to 
illustrate the relationship between classes, or assets. This is shown in Eigure 14. The 
ADAS was decomposed by the EOC, observers, and the ADAT. The operations and 
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attributes may be seen in the diagram and how they relate to the outside elass, the 
damaged airfield. 



Figure 14. Current ADAS Class Diagram 


b. Physical Architecture 

The current ADAS system consists of AD AT personnel traveling to the area of 
interest either by foot or by ground vehicle from a staging area closely located to the 
mission area. Per AFPAM 10-219 (Department of the Air Force 2008), the following 
equipment is recommended for the AD AT deployment kit: data recording and reporting 
equipment to include base grid maps, damage assessment forms, clipboards, writing 
instruments, radios, spare radio batteries, nonmetallic measuring tapes, flags and UXO 
markers. With these components in hand, the AD AT personnel would survey the area for 
what the mission calls for, make the appropriate notations, and return to the EOC to start 
the next phase of the process - entering the necessary data into GeoExPT. 
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c. CONOPS 

Figure 15 shows the CONOPS diagram for the current ADAS. The steps below 
correspond to the numbers in Figure 15. 

1. The observers at the Control tower/observation points relay initial damage 
reconnaissance to the EOC (la) and concurrently the EOC orders the AD AT 
to move from staging area to start point (lb). 

2. AD AT begins predetermined survey route to gather detailed damage and 
UXO data. 

3. AD AT visual observation and reporting of Bomb and Bomblet Eields to EOC. 

4. AD AT visual observation and reporting of UXO and Craters to EOC. 

5. AD AT visual observation and reporting of Camouflets and Spall damage to 
EOC 

6. Data reported to EOC includes; location, shape, color, markings, coordinates, 
render-safe time 
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Figure 15. Current ADAS CONOPS 


2. lADAS I 

The next step in the SE process was to define the various artifacts which would 
describe the first alternative lADAS solution. 

a. Functional Analysis 

A functional architecture was generated for the lADAS I system (see Figure 16). 
This top-down decomposition showed the functions that were performed for a notional 
lADAS I mission. Similar to the ADAS mission, the lADAS I mission is decomposed 
into the Perform Initial Reconnaissance and Compose Damage Assessment functions. 
However, with the use of the RPA, there are also Dynamic Assessment and Data 
Analysis functions. These functions are further decomposed to Observe and Perform 
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Visual Inspection, RPA Scan of Airfield Damage, RPA Scan of Airfield UXO, 
Automated Analysis, Visual Analysis, Formatting of The report, and Transmission of the 
Report. 



Figure 16. lADAS I Mission Functional Architecture Hierarchy Chart 


Figure 17 shows the EFFBD for the lADAS I. The overall mission is the same as 
the baseline ADAS; however, there is an additional step included in order to analyze the 
data being collected by the RPA. This is seen in Block 2.5 in Figure 17. The raw 
assessment data is used from the initial reconnaissance and dynamic assessment with the 
RPA and used for data analysis and to compose the damage assessment report. The data 
analysis step includes parallel efforts of computer algorithm analysis and a user visual 
analysis to determine the extent of damage and UXO. The output of the damage 
assessment report includes information of the threat such as size, damage type, location, 
and quantity. 
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Figure 17. lADAS I EFFBD 


The next step in developing the funetional architecture was to decompose all of 
the functions into the IDEFO format. The IDEEO diagram of the lADAS I shows the data 
flow, system control, and overall functional flow. Controls to the lADAS I include the 
Base Alarm, EOC Communication, Detection Algorithm, and Raw Assessment Data. The 
input include the Visual UXO Observation Data, Visual Damage Observation Data, 
Visual Airfield data, and the Programmed Elight information, while the outputs include 
the Analyzed Data, Compiled Report, Quantity, Eocation, Size, and Damage Type. The 
mechanisms included in lADAS I are Observation, Trained Personnel, Day Camera, and 
AD AT. Eigure 18 shows the IDEEO diagram of the lADAS I. 
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Figure 18. lADAS IIDEFO Diagram 


Following construction of the IDEFO diagram, it was necessary to analyze the 
timeline of events for the lADAS I functional process. The sequence of events occurs 
between the ADAT, Algorithm Processing Computer, EOC, RPA Operator, RPA Plight 
Control, and the Day Camera. The EOC initiates the ADAT Deployment Order to the 
ADAT. The ADAT then sends the Programmed Plight Path Data and the RPA Paunch 
Order to the RPA Operator to begin their mission. The flights tasks are then provided to 
the RPA Plight Control. Once in the air, the Day Camera provides a direct feed of Eive 
Imagery onboard the RPA, which is relayed as Raw Assessment Data to the ADAT and 
Algorithm Processing Computer. The computer processes this information and provides 
Algorithm Analyzed Data to the ADAT who then compile the report and provide it to the 
EOC. The sequence diagram can be seen in Pigure 19. 
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Figure 19. lADAS I Sequence Diagram 


Breaking down the activity further was a class diagram to describe the lADAS I 
classes. The classes included attributes, operations, and parameters which were used to 
illustrate the relationship between classes, or assets. This is shown in Figure 20. 



Figure 20. lADAS I Class Diagram 
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b. Physical Architecture 

There are four main components in the physical architecture for the lADAS I 
system. The main categories are: mechanism, sensor, data processor, and communication. 
Extracting the data associated with the highest score from the Pugh matrix in Table 14 , 
option 4 provides the physical architecture subcomponents for lADAS 1. The elements 
are described in Table 15. 


Table 15. lADAS 1 Physical Architecture 


Mechanism 

Sensor 

Data Processor 

Communication 

RPA (Puma AE 
RQ-20B) 

Day Camera 
(Zenmuse X5S) 

CPU 

Digital Data Eink 
(DDE) 


The Puma AE (All Environment) RQ-20B was selected based on the inherent 
capabilities of the system and how these capabilities would lend to a reasonable solution 
for the lADAS I system. This system is fully waterproof, small, and designed for land or 
maritime operations. The Puma AE has an enhanced precision navigation system with a 
secondary GPS to provide greater positional accuracy and reliability (AeroVironment 
2017). The Puma AE was upgraded with a Zenmuse X5S camera capable of capturing 21 
megapixel images in order to capture very high resolution imagery of the surveyed area 
(DJI 2017b). This camera has been successfully integrated onto other types of RPAs 
including the DJI Inspire 2 RPA to capture high resolution imagery (DJI 2017a). The 
RPA cameras in general will not be exposed to all the weather conditions like the tower 
cameras and will only be exposed to various weather conditions during flight operations. 

In searching for an appropriate CPU, the analysis led the team to specify a high- 
end computer with the versatility of a laptop. After performing an extensive search, the 
resulting selection was for a Hewlett Packard (HP) ZBook 17 G4 Mobile Workstation. 
The website (Hewlett Packard 2017) provided the ability to select the platform with the 
highest memory (16GB) along with the highest storage (1TB) in order to process large 
file sizes due to the high resolution images planned for that hardware. In addition, since 
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the primary goal of the platform would be to evaluate images, a graphics card and image 
processor with significant capability was selected. 

Once the physical architecture was determined, the next step in the process was to 
complete the CONOPS diagram, visually displaying how the components work together 
to provide the system operation. 

c. CONOPS 

Figure 21 provides the CONOPS for lADAS I. The steps below correspond to the 
numbers in Figure 21. 

1. Observers at the Observation points relay initial damage reconnaissance to 
EOC (la) and concurrently the on/remote site EOC orders dynamic 
assessment utilizing representative RPA equipped with day camera (lb) 

2. RPA day camera begins predetermined survey scan to gather detailed damage 
and UXO data 

3. RPA imagery analyzed using specific algorithms at the EOC; report on Bomb 
and Bomblet Eields 

4. RPA imagery analyzed using specific algorithms at the EOC; report on UXO 
and Craters. 

5. RPA imagery analyzed using specific algorithms at the EOC; report on 
Camouflet and Spall damage. 

6. Data reported to EOC includes; location, shape, color, markings, coordinates, 
render-safe time. The software will create a shapefile with a summary of all 
the damage and UXO locations that will be transferred via network 
connection to the GeoExPT operator for import into GeoExPT. 
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Figure 21. lADAS I CONOPS 


3. lADAS II 

The next step in the SE process was to define the various artifacts which would 
describe the second alternative lADAS solution. 

a. Functional Analysis 

A functional architecture was generated for the lADAS II system (see Figure 22). 

This top-down decomposition showed the functions that were performed for a notional 

lADAS II mission. Similar to the previous alternative, the lADAS II system decomposes 

the lADAS II mission to the same four functions: Perform Initial Reconnaissance, 

Perform Dynamic Assessment, Data Analysis, and Compose Damage Assessment. 

Further refinement and decomposition allow this mission to be performed by a Tower in 
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subsequent functions. The level three functions include Observe and Perform Visual 
Inspection, Tower Scan of Airfield Damage, Tower Scan of Airfield UXO, Automated 
Analysis, User Visual Analysis, Formatting of the Report, and Transmission of the 
Report. 
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Figure 22. lADAS II Mission Functional Architecture Hierarchy Chart 


Figure 23 shows the EFFBD for the lADAS II. The overall mission is the same as 
the baseline ADAS; however, there is an additional step included in order to analyze the 
data being collected by the Tower. This is a similar process as the lADAS I alternative. 
This is seen in Block 2.5 in Figure 23. The raw assessment data is used from the initial 
reconnaissance and dynamic assessment with the Tower and used for data analysis and to 
compose the damage assessment report. The data analysis step includes parallel efforts of 
automated analysis and a user visual analysis to determine the extent of the damage and 
presence of UXOs. The output of the damage assessment report includes information of 
the threat such as size, damage type, location, and quantity. 
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Figure 23. lADAS II EFFBD 


The next step in developing the functional architecture was to decompose all of 
the functions into the IDEFO format. The IDEEO diagram of the lADAS II shows the data 
flow, system control, and overall functional flow. Controls for the lADAS II include the 
Base Alarm, EOC Communication, Raw Assessment Data, and the Detection Algorithm. 
The input includes the Visual UXO Observation Data, Visual Damage Observation Data, 
the Predetermined Scan Area, and the Visual Airfield Data. Outputs include the Analyzed 
Data, Compiled Report, Eocation, Quantity, Size, and Damage Type. The mechanisms 
used in lADAS II are Observation, Trained Personnel, Tower, Day Camera, Computer, 
and the AD AT. Eigure 24 shows the IDEEO diagram of the lADAS II. 
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Figure 24. lADAS IIIDEFO Diagram 


Following construction of the IDEFO diagram, it was necessary to analyze the 
timeline of events for the lADAS II functional process. The functions used in the 
decomposition of the sequence are the ADAT, Algorithm Processing Computer, EOC, 
Tower, and Day Camera. The EOC sends the ADAT Deployment Order to the ADAT. 
The predetermined scan area is provided to the Tower which uses the scan coordinates to 
point the day camera. The Day Camera provides the live imagery to the Tower which 
provides this information as Raw Assessment Data to the ADAT and Algorithm 
Processing Computer. The Computer then analyzes this data to produce Algorithm 
Analyzed Data to the ADAT. The ADAT uses this information to then compile the report 
and provide it to the EOC. The sequence diagram can be seen in Eigure 25. 
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Figure 25. lADAS II Sequence Diagram 


Breaking down the activity further was a class diagram to describe the lADAS II 
classes. The classes included attributes, operations, and parameters which were used to 
illustrate the relationship between classes, or assets. This is shown in Figure 26. 
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Figure 26. lADAS II Class Diagram 


b. Physical Architecture 

Similar to the physical architecture for lADAS I, lADAS II was selected from the 
second-highest scoring alternative in the Pugh matrix in Table 14 . Option #16 was 
selected, and is described in Table 16. 


Table 16. lADAS II Physical Architecture 


Mechanism 

Sensor 

Data Processor 

Communication 

Stationary Tower 

Day Camera 

CPU 

Wireless 


The stationary towers selected are produced by Rohn Products LLC and are 
15.2 m (50 ft) free standing tower kits (25SS040) (Solid Signal, Signal Group LLC & 
Affiliates n.d.). Based on the DRM, the 1730 m (5,676 ft) runway will require 15 towers 
and the 1981.2 m (6,500 ft) runway will require 17 towers for a total of 32 towers. Each 
tower will be placed for enough away from the runway to prevent any adverse interaction 
with aircraft. 

In searching for an appropriate CPU, the analysis led the team to specify a high- 
end computer with the versatility of a laptop. After performing an extensive search, the 
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resulting selection was for an HP ZBook 17 G4 Mobile Workstation. The HP Store 
website (Hewlett Packard 2017) provided the ability to select the platform with the 
highest memory (16GB) along with the highest storage (1TB) in order to process the 
enormous images planned for that hardware. In addition, since the primary goal of the 
platform would be to evaluate images, a significant graphics card and image processor 
was selected. 

The camera selected is a 4k security camera that offers ultra-high definition (HD) 
video recording resolution and is internet protocol (IP) accessible from a network 
computer. This camera uses the “latest Progressive Scan Impact Sensor to produce 
12 megapixel video at up to 15 fps or 4k ultra HD” (CCTV Camera World Inc. 2015). 
These cameras had the highest resolution commercially available at the time of 
researching based on being able to withstand different types of continual environmental 
conditions that can be encountered in the field and are designed to be mounted on towers 
and buildings with little or no modification. An additional feature is that the 4k Ultra HD 
allows an operator to digitally zoom with “the highest amount possible before pixilation 
occurs” (CCTV Camera World Inc. 2015). This can be very useful when trying to 
identify the type of UXO or the extent of damage. 

Once the physical architecture was determined, the next step in the process was to 
complete the CONOPS diagram, visually displaying how the components work together 
to provide the system operation. 

c. CONOPS 

Figure 27 provides the CONOPS for lADAS II. The steps below correspond to 
the numbers in Figure 27. 

1. Observers at the Control tower/observation points relay initial damage 
reconnaissance to the EOC (la) and concurrently EOC orders dynamic 
assessment utilizing day camera and wireless sensors (lb) 

2. Tower day camera begins predetermined survey scan to gather detailed 
damage and UXO data 
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3. Tower imagery analyzed using specific algorithms at the EOC report on 
Bomb and Bomblet Fields 

4. Tower imagery analyzed using specific algorithms at the EOC report on UXO 
and Craters 

5. Tower imagery analyzed using specific algorithms at the EOC report on 
Camouflet and Spall damage 

6. Data reported at the EOC includes; location, shape, color, markings, 
coordinates, render-safe time. The software will create a shapefile with a 
summary of all the damage and UXO locations that will be transferred via 
network connection to the GeoExPT operator for import into GeoExPT. 
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Figure 27. lADAS II CONOPS 
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IV. 


SYSTEMS EVALUATION 


A. INTRODUCTION 

In order to best determine a viable alternative, the decomposed architectures from 
Chapter III were modeled using the ExtendSim tool and the resulting simulations run to 
estimate the MOEs for each alternative. By implementing the alternatives as ExtendSim 
models, a Monte Carlo approach could be used to allow for randomness in user behavior. 
In this chapter, the ExtendSim model analysis, model inputs, metrics, and cost analysis 
are discussed. 

Eimited validation and verification of the ExtendSim models was performed due 
to the compressed timelines of the project, lack of available information, and lack of 
funding. Since the ADAS operation is completely manual (drive to airfield location, 
dismount to damage location, manual measurement of width/depth), the team based 
assessment times on approximations for a two-man crew to measure a crater and/or 
assess UXO and move onto the next location. 

When developing the lADAS I model, the imagery collection parameters were 
heavily dependent upon the flight characteristics of the RPA, selected camera resolution, 
and system image requirements. Recommendations for image collections and calculations 
from the DroneMapper Imagery Collection Worksheet (Drone Mapper 2015b) were 
integrated into the ExtendSim model to validate the RPA and camera settings needed for 
the required imagery resolution (5 cm/pixel). This enhanced the confidence in the lADAS 
I model with respect to the times allocated to collect the required imagery based on the 
selected UAS and camera combinations. Using free Drone Mapper RAPID 
photogrammetric imagery processing software (Drone Mapper 2015a) and sample data 
provided by Drone Mapper (Drone Mapper 2017), the sample data was processed. Even 
though the software did have limitations, the time measured to process the sample 
imagery provided confidence that the lADAS I image processing times input into the 
model appeared to be feasible. No similar software was available to provide an estimation 
for the time required to process the imagery for the lADAS II system. The increased 
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difficulties in processing the poorer images due to the poor angle and less camera 
resolution were reflected when modelling the lADAS II model image processing time. 

B. CURRENT ADAS 

1. ExtendSim Model Analysis 

Discrete event simulation and analysis of the collected simulation data was used 
to evaluate the estimated effectiveness of the current ADAS. The current ADAS was 
modeled using ExtendSim Software (see Figure 28 and Figures C-4 through C-6) . The 
ExtendSim model was based on the process outlined from the functional analysis and 
CONOPS found in Chapter III. 

The model simulated a single ADAT travelling in an HMMWV detecting, 
classifying, measuring, and reporting runway damage and UXO along a predetermined 
route. In addition to the time required to perform the assessment, the simulation also 
incorporated the time delay for the ADAT team to travel from their staging location to 
the start of the predetermined survey route. The staging area and predetermined 
assessment route is shown in Figure 29 . 
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Simulation Inputs 


Simulation Results (Last Run Only) 
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Figure 28. Current ADAS ExtendSim Model 
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Figure 29. ADAT Travel Route. Adapted from Aircraft Owners and Pilots 

Association (2017). 


The damage and unexploded ordnance incorporated into the model was based on 
the DRM outlined previously and is summarized in Table 17. Because the DRM is based 
on an expeditionary attack, the intent is to focus on the bare minimum runway surfaces 
necessary to resume sortie traffic. It should be noted that assessing all taxiways and ramp 
areas would increase the total time to complete a thorough assessment. 


Table 17. Model Inputs: DRM Damage and UXO Types and Quantities 


Damage/UXO Type 

Quantity 

Small Craters 

15 

Large Craters 

4 

Spall Fields 

1 

UXO Bomblets 

13 
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The model as built allowed for the input of six different types of damage and 
three types of UXO. The probability of detection, classification time, and measurement 
time for each damage and UXO type were estimated based on input from subject matter 
experts and engineering judgement and shown in Table 18. 


Table 18. Model Inputs: Damage and UXO Detection and Measurement 


Damage/ 
UXO Type 

Description 

Probability 

of 

Detection 

Classification Time 
(min) 

Measurement Time 
(min) 

Dist. 

Mean 

SD 

Dist. 

Mean 

SD 

Small Crater 

<10 foot 
diameter 

0.95 

Normal 

0.5 

0.1 

Normal 

2 

0.33 

Large Crater 

>10 foot 
diameter 

1 

Normal 

0.5 

0.1 

Normal 

10 

2 

Spall 

Spall type 
damage 

0.95 

Normal 

0.5 

0.1 

Normal 

2 

0.33 

Camouflet 

Camouflet type 
damage 

0.9 

Normal 

0.5 

0.1 

Normal 

1 

0.25 

Crater Field 

Field of 20+ 
craters located 
in close 
proximity 

1 

Normal 

0.25 

0.1 

Normal 

15 

2 

Spall Field 

Field of 20+ 
spalls located in 
close proximity 

1 

Normal 

0.25 

0.1 

Normal 

10 

2 

UXO- 

Bomblet 

Submunition 

0.9 

Normal 

2 

0.25 

Normal 

3 

1 

UXO-Bomb 

Large aircraft 
type bomb 

0.95 

Normal 

2 

0.5 

Normal 

2 

0.5 

UXO-Round 

Mortar/Artillery 

Round 

0.9 

Normal 

2 

0.25 

Normal 

3 

1 

NOTES: 

SD - Standard Deviation 

Dist - Distribution Type 


Travel time for the ADAT team from the staging location to the start of the 
predetermined route and the rate at which ADAT team could travel and visually search 
for damage/UXO during the actual assessment were also estimated and input into the 
model (see Table 19). 
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Table 19. Model Inputs; Staging Time, Assessment/Travel Speed, and 

Reporting Times 


Input 

Description 

Distribution 

Mean 

Standard 

Deviation 

Staging Travel Time 
(minutes) 

Time to travel from 
staging area to start of 
damage assessment 
route 

Normal 

10 

2 

AD AT Travel 
Assessment Speed 
(miles per hour) 

Average speed of 
vehicle during damage 
assessment 

Normal 

20 

2 

Assessment Report 
Time (minutes) 

Time to transmit 
damage/UXO report at 
each location 

Triangular 

0.5 minimum, 1.5 
Maximum, 1 most 
likely 


Several MOEs were outputs of the simulation model and were used to evaluate 
the effectiveness of the current ADAS method. These MOEs were selected to allow a 
comparison of the effectiveness between the three systems. The selected MOEs used 
throughout the simulation and analysis are presented in Table 20 and described in detail. 


Table 20. Current ADAS Effectiveness/Performance Measures and 

Metrics 


MOEs 

Metrics 

% Airfield Damage Assessed 

Airfield Damage Assessed/Total Airfield Damage 

% UXO Assessed 

UXO Detected/Total Airfield UXO 

Airfield Damage Assessment 
Time 

Time from end of airfield attack to completion of ADA 

Travel/Detection Time 

Total AD AT time spent travelling/detecting damage 
and UXO 

Classification Time 

Total AD AT time spent classifying damage and UXO 

Measure Time 

Total AD AT time spent measuring and locating damage 

Communication Time 

Total AD AT time spent communicating damage results 
to EOC 


% Airfield Damage and % UXO Assessed . The mission of the ADA is to provide 
a detailed damage and UXO assessment to the EOC to support determination of repair 
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efforts, therefore, these are the primary MOEs of concern. Failure to assess the majority 
of damage/UXO results in mission failure. 

Airfield Damage Assessment Time. The mission of the ADA is to provide a 
detailed damage and UXO assessment as quickly as possible so runway repair activities 
can commence. A faster ADA time provides an indication that one system may be more 
capable than another. 

Travel/Detection Time. This MOE estimates how much time it takes to transit the 
assessment route scanning for damage. A faster travel/detection provides an indication 
that one system may be more capable than another. 

Classification Time. This MOE estimates how much time it takes to classify 
damage and UXO. A faster classification time provides an indication that one system 
may be more capable than another. 

Measure Time. This MOE estimates how much time it takes to determine the 
location and measure the size of the damage or UXO. A faster measure time provides an 
indication that one system may be more capable than another. 

Communication Time. This MOE estimates how much time it takes to 
communicate the location and the size of the damage or UXO. A faster measure time 
provides an indication that one system may be more capable than another. 

The model was run 500 times for statistical significance and to also model system 
variability. The collected simulation data was analyzed at the conclusion of the 
simulation and the results are shown in Table 21. 
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Table 21. Current ADAS Effectiveness/Performance Measures 


Measure 

Average 

95% 

Confidence 

Interval 

Min 

Max 

% Damage Assessed* 

95.3% 

[ 94.9,95.7 ] 

73.7 

100.0 

% UXO Assessed* 

95.4% 

[ 94.8,95.9 ] 

71.4 

100.0 

Airfield Damage Assessment 

Time* 

174.2 

[ 173.5 , 175 ] 

147.2 

195.3 

Staging Transit Time (minutes) 

10.0 

[9.9,10.2] 

3.2 

16.1 

Assessment Travel/Detection 

Time (minutes) 

6.7 

[ 6.7,6.8 ] 

5.8 

8.0 

Assessment Classification Time 
(Minutes) 

31.3 

[31.1 ,31.4] 

23.4 

35.3 

Assessment Measure Time 
(minutes) 

94.6 

[94.1,95.2] 

68.5 

111.5 

Assessment Comm Time 
(minutes) 

29.3 

[29.1,29.4] 

26.3 

35.8 

NOTE: * Measure of Effectiveness as described in Table 9 


The effectiveness of the AD AT team in assessing UXO and damage is very high 
which is not surprising as the damage and UXO in the DRM should not be hard to 
identify with the AD AT team surveying damage from very close distances. Also 
expected was the long overall duration of the assessment which is due to the very low 
automation and serial process in which the process must be conducted. 

2. Cost Analysis 

The first step in determining the cost associated with the current ADAS, was to 
determine which costs were considered. Using the cost methodology previously defined, 
the costs were: R&D, SE, Personnel, and O&S. Both the R&D and SE costs were $0 for 
the ADAS system. The reasoning for his was based on the fact that the current system is a 
completely manual process. Personnel costs include barracks and base salary for five 
personnel. Since these are shared resources, only a portion of the total personnel cost is 
considered for ADAS purposes. The O&S cost includes annual training, transport, 
maintenance, and fuel. Table 22 breaks down the estimated costs associated with ADAS. 
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Table 22. ADAS Cost Estimate 


Type of Cost 

Description 

Cost 

R&D 

Shared between all Resources 

$0 

SE 

System Engineering expenses for defining the 
system 

$0 

Personnel 

Barracks = $1.2M, 10% for AD AT 

$120K 

Salary = $500K/yr, 20% for ADAT over 
10 yrs 

$1000K 

O&S 

Transportation 

HMMWV = $220K, 20% for ADAT 

$44K 

Maintenance 

Annual Training = $500K, since same for all 
three options for personnel ($1 OK/year/ 

ADAT team member - five personnel per 
team) 

$500K 

Transport Maintenance = According to a 
RAND study (Pint, et al. 2008) the cost is 
about $5.53/mile. Estimate lOK miles/year = 
$55.2K/year for maintenance. Ten years = 
$550K, 20% for ADAT 

$110K 

Fuel = HMMWV average 8 MPG on the 
highway and 4 MPG in the city (Richard 
2008). Using the average of 6 MPG, and 20% 
of lOK miles/year = 2K miles. For 10 years 
that is 20K miles. At 6 MPG, that is about 
3,334 gallons of diesel fuel. In today’s dollars 
that is about $2.50/gallon = $8K 

$8K 

Total ADAS Cost 

$1782K 


C. lADAS I 

1. ExtendSim Model Analysis 

Discrete event simulation and analysis of the collected simulation data was used 
to evaluate the estimated effectiveness of the first lADAS alternative system. The first 
system alternative was modeled using ExtendSim Software (see Figure 30 and Figures 
C-7 through C-9) and was based on the process outlined from the functional analysis and 
CONOPS found in Chapter III. 
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Figure 31. lADAS I ExtendSim Model 
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As shown in Figure 31, the model simulated an RPA collecting overhead imagery 
of the runway surfaces that was then transmitted over a digital data link to the RPA 
ground control station for subsequent analysis by the ADAT. The ExtendSim model 
incorporated the follow activities and sequence: 

• the RPA was launched and travelled from launch site to beginning of 
predetermined survey route 

• the RPA flew predetermined route collecting overhead imagery at 
predetermined intervals and transmitted imagery via digital data link to the 
RPA ground control station 

• the imagery was processed into a single orothomosaic image by the 
ADAT 

• the imagery was analyzed by the ADAT using software that automatically 
detected, classified, and measured damages 

• verification of the automated damage assessment was performed manually 
by ADAT personnel to verify results and eliminate false detections 

• manual review of the orthomosaic image was conducted by ADAT 
personnel to identify damage that was missed by the automated damage 
assessment 
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Figure 32. lADAS I Sequence of Events. Adapted from Aircraft Owners and 

Pilots Association (2017). 


The quantity and location of damage and UXO incorporated into the model was 
based on the DRM outlined previously and is summarized in Table 23. 


Table 23. lADAS 1 Model Inputs: DRM Damage and UXO Types and 

Quantities 


Damage/UXO Type 

Quantity 

Small Craters 

15 

Large Craters 

4 

Spall Fields 

1 

UXO Bomblets 

13 


The model as built allowed for the input of different types of damage, RPA flight 
characteristics, camera characteristics, detection probabilities, and estimated activity 
times as shown in Tables 24 through 27. The camera specifications and imagery 
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collection parameters were chosen based on achieving a ground sampling distance of 
5 cm/pixel in order to maximize the probability of detecting very small UXO bomblets. 


Table 24. lADAS I Model Inputs: RPA Camera Specifications 


RPA Camera Specifications 

Image (Pixel Width) 

5280 

Image - (Pixel Height) 

3956 

Focal Plane Width (mm) 

18 

Focal Plane Height (mm) 

13.5 

Lens Focal Length (mm) 

10 


Table 25. lADAS I Model Inputs: RPA Imagery Collection 


Imagery Collection 

RPA Ground Speed (mph) 

30 

Flight Height (m) 

30 

Area Survey Length (m) per run 

2134 

Area Survey Width (m) per run 

305 

Forward Image Overlap (%) 

60 

Side Image Overlap (%) 

40 


79 






Table 26. lADAS I Model Inputs: Activity/Parameters 


Activity/Parameter Distribution Mean SD 

RPA Launch and Transit Parameters 

RPA Prep and Launch Time (minutes) 

Normal 

5 

1 

Distance - Staging Area to Start of Run (meters) 

Constant 

2000 

N/A 

RPA Transit Speed from Staging Area (miles per hour) 

Constant 

30 

N/A 

Automated Software Processing Times 

Orthomosaic Creation (minutes) 

Normal 

15 

2 

Auto detection of Damage/UXO (minutes) 

Normal 

5 

1 

Verification of Auto Detection Result Timelines 

Large Crater (minutes) 

Normal 

0.08 

0.02 

Small Crater (minutes) 

Normal 

0.08 

0.02 

Camouflets (minutes)) 

Normal 

0.33 

0.04 

Craterfield (minutes) 

Normal 

0.16 

0.02 

Spallfield (minutes) 

Normal 

0.16 

0.02 

UXO - Bomb (minutes) 

Normal 

0.33 

0.04 

UXO - Bomblet (minutes) 

Normal 

0.5 

0.1 

UXO - Round (minutes) 

Normal 

0.33 

0.04 

Spall (minutes) 

Normal 

10 

0.02 

Manual Review of Imagery 

Review of Imagery (minutes) 

Normal 

5 

1 

Individual Damage/UXO Location Classification Time 
(minutes) 

Normal 

0.5 

0.1 

Individual Damage/UXO Location Measurement Time 
(minutes) 

Normal 

0.5 

0.5 
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Table 27. lADAS I Model Inputs: Automated and Manual Detection 

Probabilities 


Damage/UXO Type 

Description 

Automated 

Manual 

Small Crater 

< 3 m (10 ft) diameter 

0.8 

0.8 

Large Crater 

> 3 m (10 ft) diameter 

0.9 

0.8^ 

Spall 

Spall type damage 

0.85 

0.8 

Camouflets 

Camouflets type damage 

0.7 

0.8 

Crater Field 

Field of 20+ craters located in close 
proximity 

0.95 

0.9 

Spall Field 

Field of 20+ spalls located in close 
proximity 

0.95 

0.9 

UXO - Bomblet 

Submunition 

0.7 

0.8 

UXO- Bomb 

Large aircraft type bomb 

0.6 

0.6 

UXO - Round 

Mortar/Artillery Round 

0.7 

0.8 

NOTE: Probabilities associated with manual review of imagery reflect the operator 
performing a quick review of the imagery versus a detailed assessment. 


Outputs from the simulation model were utilized to evaluate the effectiveness of 
the lADAS I system. The selected measures used throughout the simulation and analysis 
are presented in Table 28 and described in detail. 
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Table 28. lADAS I Simulation Outputs 


MOEs 

Metrics 

% Damage Assessed* 

Airfield Damage Detected/Total Airfield Damage 

% UXO Assessed* 

UXO Detected/Total Airfield UXO 

Airfield Damage Assessment 
Time* 

Time from end of airfield attack to completion of 
airfield damage assessment 

Staging Transit Time (minutes) 

Time from end of airfield attack to time RPA arrives 
at start of survey route 

Survey Time (minutes) 

Total RPA time spent collecting imagery data 

Image Processing Time (minutes) 

Total software runtime processing images into 
orthomosaic 

Auto detection Run Time 
(minutes) 

Total Software runtime for auto detection of damage/ 
UXO 

Verification Time (minutes) 

Total time verifying auto detection results and 
elimination of false detections 

Manual Scan Time (minutes) 

Total time for operator to review orthomosaic for 
damage not identified by auto detection software. 

Manual Classification Time 
(minutes)) 

Total time for manual classification of damage/UXO 

Manual Measurement Time 
(minutes) 

Total time for measurement of damage/UXO 

Note: * Measure of Effectiveness as described in Table 9 


The model was run 500 times for statistical significance and to also model system 
variability. The collected simulation data was analyzed at the conclusion of the 
simulation and the results are shown in Table 29. 
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Table 29. lADAS I Simulation Output Results 


Output 

Average 

95% 

Confidence 

Interval 

Min 

Max 

% Damage Assessed* 

96.8% 

[96.5,97.1 ] 

80.00% 

100.00% 

% UXO Assessed* 

83.9% 

[ 83,84.8 ] 

46.15% 

100.00% 

Airfield Damage Assessment 

Time* 

52.07 

[51.8,52.3] 

41.9 

60.9 

Staging Transit Time (minutes) 

7.5 

[7.4,7.6] 

4.4 

9.9 

Survey Time (minutes) 

8.4 

[8.4,8.4] 

7.8 

8.7 

Image Processing Time (Minutes) 

15.1 

[ 14.9,15.2] 

8.6 

19.9 

Autodetection Run Time 
(minutes) 

5.0 

[4.9,5.1] 

1.7 

7.9 

Verification Time (minutes) 

5.3 

[5.3,5.4] 

2.8 

8.3 

Manual Scan Time (minutes) 

5.0 

[4.9,5.1] 

2.3 

8.1 

Manual Classification Time 
(minutes) 

2.9 

[2.8,3] 

0.5 

6.5 

Manual Measurement Time 
(minutes) 

2.9 

[2.8,3] 

0.4 

6.6 

NOTE: * Measure of Effectiveness as describee 

in Table 9 


The effectiveness of the lADAS I system in assessing UXO and damage was 
greater than 80%. A lower percentage of total UXO was detected when compared to 
damage, which was expected due to the much smaller size of the UXO bomblet dud 
munitions (less than 6 in) in the DRM. 

2. Cost Analysis 

The cost analysis for the first lADAS alternative was broken down into four 
components: R&D, SE, Personnel, and O&S costs. Each cost component contributed to 
the overall cost of designing, developing, and maintaining the system through ten years 
of maintenance and operations. The summation of those costs provided the stakeholders 
with an understanding of the 10-year system life cycle cost. The R&D, along with the SE 
cost analysis, was performed using COSYSMO. A Person-Month of $10K/month was 
used. Eor the software cost model, an analogous model to other academic exercises using 
RPAs was used. An estimate of 11,000 new Source Eines of Code (SEOC) 4,000 reused 
SEOC, 25% integration required, and 3% Assessment and Assimilation values were used. 
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The detailed description for each category is captured in Table 30. Screenshots of the 
COSYSMO tool are shown in Appendix C, Figures C-10 through C-13 . 


Table 30. lADAS 1 COSYSMO SE Inputs/Assumptions 


# Requirements = 

11 Functional Requirements 

Three Easy 

Six Nominal 

Two Difficult-Detect Camouflet, ID UXO 

# System Interfaces = 

Three System Interfaces 

Difficult-AD AT to RPA via remote control 

Nominal-RPA to CPU via wireless 

Nominal-CPU to EOC via wireless 

# Algorithms = 

Seven Algorithms 

Easy-Flight operations 

Nominal-Image Capture 

Nominal-Data Transfer 

Nominal-Crater Size Determination 

Nominal-Crater Location Determination 

Difficult-UXO Type Determination 

Nominal-UXO Location Determination 

# Operational Scenarios = 

Two Operational Scenarios 

Difficult-UXO Classification 

Nominal-Damage Determination 


The Software Cost Drivers were set to “Nominal” with the following exceptions: 

• the Requirements Understanding, Architecture Understanding, and 
Stakeholder Team Cohesion parameters were set to “High,” 

• the # of Recursive Levels in the Design parameter was set to “Low,” and 

• the Multisite Coordination parameter was set to “Very Low.” 

The resulting cost estimate for the first two components of lADAS I (Systems 
Engineering and Software Development) is shown in Figure 33. 
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Components 


RPA 

Dav Came'a 

W 'eess Commun'caton 

CPU 



System Emnneerir« cost 

EffO't 

21.7 

Pe'son-Months 

Sch€<Ju€ 

4.1 

Months 

Cost 

5216.926 



Maintenance Cost 

Annua Ma'ntenance EffO't 

1.9 

Pe'son-Months 

Annua Ma'ntenance Cost 

515.593 


Tota Ma'ntenance Cost {10 Y^) 

5155.935 



Software Costs 

EffO't 

42.5 

Pe'son-months 

ScheOue 

12.7 

Months 

Cost 

5425.015 



Software MaintenarKe Cost 

Annua Ma'ntenance EffO't 

3.2 

Pe'son-months 

Annua Ma'ntenance Cost 

535.513 


Tota Ma'ntenance Cost {10 Y^) 

5355.137 



Total Cost, Option 2 

System Eng*nee'‘'ng -►Sofnva'e Cost A Ma'ntenance = 

51,222,016 


Figure 33. lADAS I Cost Modeling Summary 
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The next cost that was defined in the cost model was for personnel. A two-man 
crew is needed to operate a Puma. The other component requiring personnel support 
would be the manual confirmation of the analysis performed by the software. This would 
add an additional two resources to the personnel total, which brought it to a total of four 
personnel. 

The final component for the cost analysis was the O&S costs. Table 31 contains 
the results of the search for the necessary components. 


Table 31. lADAS I O&S Costs 


Item 

Cost 

Picture 

RPA (Puma AE 

$250,000 

1 1 
- 9 ft -►* 

RQ-20B) 

(Aero V ironment 

1 1 

1 « 1 


2017) 

i ■ i 


Complete air 
system includes 
three air 
vehicles, two 

» ( 


ground control 
stations, and 
support 

A 


equipment 

- — 4.6 ft -►! 

1 1 


Continued next page. 
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Table 31. 

Cost 


Continued from previous page. 

Picture 




Maintenance 
(RPA - 10 years 
- three units) 
(CPU - 10 year - 
two units) 
(Camera - 10 
year - three units) 
Total 


$125,000 

$4,163 

$ 10,000 

$2,944K 


Using a similar table from the total costs for the ADAS system, lADAS I costs 
have been summarized in the Table 32. 
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Table 32. lADAS I Cost Estimate 


Type of Cost 

Description 

Cost 

R&D 

Software to perform automated analysis 

$428K 

SE 

System Engineering expenses for defining the 
system 

$217K 

Personnel 

Barracks = $1.2M, 10% for AD AT 

$120K 

Salary = $400K/yr, 20% for AD AT over 10 yrs 

$800K 

O&S 

Hardware 

PUMA and Faptop 

$257K 

Maintenance 

Annual Training = $400K, since same for all 
three options for personnel ($1 OK/year/AD AT 
team member - four personnel per team) 

$400K 

RPA 10 Year Maintenance 

$125K 

CPU 10 Year Maintenance 

$4K 

SW 10 Year Maintenance 

$388K 

SE 10 Year Maintenance 

$189K 


The total expense for lADAS I was $2,928,000. 


D. lADAS II 

I. ExtendSim Model Analysis 

Discrete event simulation and analysis of the collected simulation data was used 
to evaluate the estimated effectiveness of the second lADAS alternative system. The 
system alternative was modeled using ExtendSim Software (see Figure 34 and Figures 
C-14 through C-16) and was based on the process outlined from the functional analysis 
and CONOPS found in Chapter III. 
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The model simulated imagery being collected by static cameras mounted on 32 
observation towers to collect imagery of the runway surfaces that was then transmitted 
over a wireless network to the AD AT processing computer for subsequent analysis by the 
AD AT. The ExtendSim model incorporated the follow activities and sequence: 

• thirty-two cameras mounted on 50 ft. towers would take an image of a 
given length of runway (400 ft. per camera) 

• image data from each cameras was transmitted via wireless network to the 
AD AT processing computer 

• imagery was processed by the computer to allow geospatial data to be 
extracted from image 

• the imagery was analyzed by the AD AT using software that automatically 
detected, classified, and measured damage 

• verification of the automated damage assessment was performed manually 
by AD AT personnel to verify results and eliminate false detections 

• manual review of the imagery was conducted by ADAT personnel to 
identify damage that was missed by the automated damage assessment 

The damage and UXO incorporated into the model was based on the DRM 
outlined previously and is summarized in Table 33. 


Table 33. lADAS II Model Inputs: DRM Damage and UXO Types and 

Quantities 


Damage/UXO Type 

Quantity 

Small Craters 

15 

Large Craters 

4 

Spall Fields 

1 

UXO Bomblets 

13 
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The model as built allowed for the input of different types of damage, image size 
of 12 mega pixels, detection probabilities, and estimated activity times as shown in 
Tables 34 and 35. Some activity times and probabilities of detection vary between the 
lADAS I and lADAS II systems. The imagery taken by the lADAS I system is of higher 
quality due to better camera image resolution and benefits from an optimal camera 
viewing angle as the RPA flies directly over the target and captures imagery 
orthogonally. In contrast, the lADAS II system utilized a camera with less image 
resolution and is mounted on a tower at a 30-45 degree viewing angle. 


Table 34. lADAS II Model Inputs: Activity/Parameters 


Activity/Parameter Distribution Mean SD 

Wireless Network 

Average Wireless Data Speed (Mb/S) 

Uniform 

10 

20 

Automated Software Processing 

5 Times 

Geospatial Processing (minutes) 

Normal 

15 

2 

Auto detection of Damage/UXO (min) 

Normal 

10 

1 

Verification of Auto Detection Result Timelines 

Large Crater (min) 

Normal 

0.08 

0.02 

Small Crater (min) 

Normal 

0.08 

0.02 

Camouflet (min) 

Normal 

0.33 

0.04 

Craterfield (min) 

Normal 

0.16 

0.02 

Spallfield (min) 

Normal 

0.16 

0.02 

UXO - Bomb (min) 

Normal 

0.33 

0.04 

UXO - Bomblet (min) 

Normal 

0.5 

0.1 

UXO - Round (min) 

Normal 

0.33 

0.04 

Spall (min) 

Normal 

10 

0.02 

Manual Review of Imagery 

Review of Imagery (min) 

Normal 

10 

1 

Individual Damage/UXO Location Classification Time 
(min) 

Normal 

0.5 

.05 

Individual Damage/UXO Location Measurement Time 
(min) 

Normal 

0.5 

0.1 
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Table 35. lADAS II Model Inputs: Automated and Manual Detection 

Probabilities 


Damage/UXO Type 

Description 

Automated 

Manual 

Small Crater 

< 3 m (10 ft) diameter 

0.7 

0.75 

Large Crater 

> 3 m (10 ft) diameter 

0.8 

0.75 

Spall 

Spall type damage 

0.5 

0.6 

Camouflets 

Camouflets type damage 

0.5 

0.5 

Crater Field 

Field of 20+ craters located in close 
proximity 

0.8 

0.8 

Spall Field 

Field of 20+ spalls located in close 
proximity 

0.8 

0.8 

UXO - Bomblet 

Submunition 

0.6 

0.7 

UXO- Bomb 

Large aircraft type bomb 

0.4 

0.5 

UXO - Round 

Mortar/Artillery Round 

0.5 

0.6 

NOTE: The probabilities are lower with lADAS II when compared to lADAS I due to 
lower quality imagery with respect to resolution and angle. 


Outputs from the simulation model were utilized to evaluate the effectiveness of 
the lADAS II system. The selected measures used throughout the simulation and analysis 
are presented in Table 36 and described in detail. 
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Table 36. lADAS II Simulation Outputs 


MOEs 

Metrics 

% Damage Assessed 

Airfield Damage Detected/Total Airfield Damage 

% UXO Assessed 

UXO Detected/Total Airfield UXO 

Airfield Damage Assessment 
Time (minutes) 

Time from end of airfield attack to completion of 
airfield damage assessment 

Image Data Transmit Time 
(Minutes) 

Time to transmit all camera images 

Image Processing Time 
(Minutes) 

Total software runtime processing images to extract 
geospatial data 

Auto detection Run Time 
(minutes) 

Total Software runtime for auto detection of damage/ 
UXO 

Verification Time (minutes) 

Total time verifying auto detection results and 
elimination of false detections 

Manual Scan Time (minutes) 

Total time for operator to review imagery for damage 
not identified by auto detection software. 

Manual Classification Time 
(minutes) 

Total time for manual classification of damage/UXO 

Manual Measurement Time 
(minutes) 

Total time for measurement of damage/UXO 

NOTE: * Measure of Effectiveness as described in Table 9 


The model was run 500 times for statistical significance and to also model system 
variability. The collected simulation data was analyzed at the conclusion of the 
simulation and the results are shown in Table 37. 
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Table 37. lADAS II Simulation Output Results 


MOEs 

Average 

95% 

Confidence 

Interval 

Min 

Max 

% Damage Assessed* 

93.0 

[92.5,93.5 ] 

75.0 

100.0 

% UXO Assessed* 

70.3 

[69.2,71.5] 

23.1 

100.0 

Airfield Damage Assessment Time 
(minutes) 

47.2 

[ 46.9,47.5 ] 

37.8 

56.2 

Image Data Transmit Time 
(minutes) 

0.3 

[ 0.3,0.3 ] 

0.2 

0.4 

Image Processing Time (Minutes) 

15.0 

[ 14.8 , 15.2] 

8.2 

22.2 

Autodetection Run Time (minutes) 

10.0 

[9.9,10.1 ] 

7.1 

12.6 

Verification Time (minutes) 

3.9 

[3.8,4] 

1.3 

6.8 

Manual Scan Time (minutes) 

10.0 

[9.9,10.1 ] 

6.9 

12.6 

Manual Classification Time (minutes) 

4.0 

[3.9,4.1] 

0.4 

8.0 

Manual Measurement Time (minutes) 

4.0 

[3.9,4.1] 

0.4 

7.6 

NOTE: *Measure of Effectiveness as described in Table 9 


2. Cost Analysis 

The cost analysis for the second lADAS alternative was broken down into the 
same four components: R&D, SE, personnel, and O&S costs. The SE effort was 
performed using COSYSMO. The inputs and assumptions entered into the model were as 
shown in Table 38. Screenshots of the COSYSMO tool are shown in Appendix C, 
Eigures C-17 through C-19 . The number of personnel necessary to support the lADAS II 
concept was reduced, as compared to both ADAS and lADAS 1. In the lADAS II 
concept, there were only two ADAT personnel needed. The reasoning was that the 
system was nearly completely automated, and that there was only the requirement for 
manual validation of the damage size, damage location, UXO identification, and UXO 
location after the software performed its function. 
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Table 38. lADAS II COSYSMO SE Inputs/Assumptions 


# Requirements = 

11 Functional Requirements 

Three Easy 

Six Nominal 

Two Difficult-Detect camouflets, ID UXO 

# System Interfaces = 

Two System Interfaces 

Nominal-Stationary Tower to CPU via Wireless 

Nominal-CPU to EOC via wireless 

# Algorithms = 

Six Algorithms 

Nominal-Image Capture 

Nominal-Data Transfer 

Nominal-Crater Size Determination 

Nominal-Crater Location Determination 

Difficult-UXO Type Determination 

Nominal-UXO Location Determination 

# Operational Scenarios = 

Two Operational Scenarios 

Difficult-UXO Classification 

Nominal-Damage Determination 


A Person-Month of $10K/month was used. For the software cost model, an 
estimate of 4,000 new SLOC 2,000 reused SLOC, 25% integration required, and 3% 
Assessment and Assimilation values were used with all software scale drivers set to 
“Nominal.” The Software Cost Drivers were set to “Nominal” with the following 
exceptions: 

• the Required Software Reliability, Developed for Reusability, and Use for 
Software Tools parameters were set to “High,” 

• the Database Size, Product Complexity, and Platform Volatility 
parameters were set to “Low,” and 

• the Multisite Development parameter was set to “Very Low.” 

Maintenance considerations included annual change size of 300 Equivalent SLOC 
(ESLOC)/year, software understanding set to 35%, and software unfamiliarity set to 0.2. 
The resulting cost estimate for lADAS II is shown in Figure 35. 
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Components 

Staton a To^s'e's 
Day Carnes 

W'^eess Common caton 

CPU 


System Ei^ineerii^ Cost 

EffoT 

13.3 

Pe'son-Months 

Sche<Ju € 

3.5 

Months 

Cost 

S133.0S5 



Maintenance Cost 

Annua Maintenance EffoT 

1.2 

Pe'son-Monttis 

Annua Ma'ntenance Cost 

$11,591 


Tota Maintenance Cost (10 Y") 

S115.912 



Software Costs 

EffOT 

12.6 

Pe'son-months 

Schedue 

5.5 

Months 

Cost 

S125.726 



Software Maintenance Cost 

Annua Maintenance Effo't 

0.7 

Pe'son-months 

Annua Maintenance Cost 

S7.418 


Tota Ma'nenance Cost (10 yr) 

574.154 



Total Cost. Option 1 

System Engineering -►Sofriva'e Cost + A Maintenance = 

$445,907 


Figure 35. lADAS Option II Cost Modeling Summary 
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The final cost component for lADAS II was the summary of hardware 
components which were brought together to reflect those costs. Table 39 summarizes the 
costs. The use of stationary towers required assumptions on the quantity. After reviewing 
the requirements, towers were specified to be 121.9 m (400 ft) apart, at a height of 
15.2 m, (50 ft), so that the color cameras had an adequate view of the area to be 
inspected. With the DRM specifying a 1730 m (5,676 ft) runway and 1987.6 m (6,521 ft) 
runway, this required a total of 32 towers to cover the necessary surface area. The all- 
weather day camera would then be suspended from the height of 15.2 m, (50 ft). One 
additional component was added from a maintenance perspective, and that was a 
motorized cherry picker so that a crew could access those cameras for repair and/or 
replacement. The total expense for lADAS II was $1,436,000. Table 40 provides the 
lADAS II Cost Estimate. 


Table 39. lADAS II O&S Costs 


Item 

Cost 

Picture 

Stationary Towers 

$29,920 



($880 each) (Solid 


A « 


Signal, Signal 

32 

j ' 


Group EEC & 

towers 

A. « 

'a\ 


Affiliates n.d.) 

with two 

4 - 



spares 

L» 




i' 




i 


Tower Installation 

$51,200 


($1,600 each, 8 



hours assembly. 



four personnel. 



guy-wire 



attachment, and 



securing) 




Continued next page. 
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Talbe 39. Continued from previous page. 


Item 


Cost 


Picture 


Cherry Picker 
(maintenance) 
(Aerial Titans 
n.d.) 


$62,900 






r’ 

* . « 




Day Cameras 
($1,196 each) 
(CCTV Camera 
World Inc. 2015) 


$47,838 
with eight 
spares 



• • % 


CPU (Hewlett 
Packard 2017) 


$7,278 
with one 
spare 



Maintenance 
(Towers, 10 yr) 
(Day Cameras, 
10 Yr) 

(CPU, 10 yr) 


$11,968 

$47,838 

$7,278 


Total 


$224,834 
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Table 40. lADAS II Cost Estimate 


Type of Cost 

Description 

Cost 

R&D 

Software to perform automated analysis 

$126K 

SE 

System Engineering expenses for defining the 
system 

$133K 

Personnel 

Barracks = $1.2M, 10% for AD AT 

$120K 

Salary = $200K/yr, 20% for ADAT over 
10 yrs 

$400K 

O&S 

Hardware 

Towers & Installation 

$81K 

Cherry Picker 

$63K 

Day Cameras 

$48K 

CPU 

$7K 

Maintenance 

Annual Training = $200K, since same for all 
three options for personnel ($1 OK/year/ 

ADAT team member - two personnel per 
team) 

$200K 

Tower 10 Year Maintenance 

$12K 

Camera 10 Year Maintenance 

$48K 

CPU 10 Year Maintenance 

$7K 

SW 10 Year Maintenance 

$75K 

SE 10 Year Maintenance 

$116K 

Total lADAS II Cost 

$I436K 


The previous set of data compared the capabilities and cost structure for ADAS, 
lADAS I, and lADAS 11. The next section of the report contains the conclusions drawn 
from the overall analysis of those resulting data points, and provides a recommendation 
for which system should be pursued for future consideration. 
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V. 


CONCLUSIONS AND RECOMMENDATIONS 


A. PROJECT SUMMARY 

The overall goal of this report was to investigate the subsystem components that 
could be brought together in order to meet the requirements for an ADA, build the cost 
model relevant to the system as a whole, and model the performance to determine the 
timeframe necessary to complete the mission for a specific DRM. The first step was to 
define the requirements necessary to meet the needs of an ADAT during the course of 
executing their mission. The ADAT mission was decomposed into a few simple steps: 
travel to the site of interest, assess damage site size and location, assess UXO location 
and identification, and complete the mission by reporting the data to the EOC. 

Once the requirements were determined, the next step was to investigate the 
physical components that could be utilized to create an autonomous system. A 
morphological box was used to create a table of the elements that would be necessary to 
complete the system concept. This resulted in 72 different possible combinations. 
Through a down select procedure using rationale like “a satellite will not be hardwired 
for communications” and “it is unlikely to replace every runway with embedded 
sensors,” the possibilities were reduced to 20 options. Using the Pugh matrix, comparing 
certain options against a known baseline, the top two ranked system configurations were 
selected. 

Several aspects for each system description were defined: functional analysis, 
physical architecture, and CONOPS. The functional analysis was the most detailed, since 
several artifacts were necessary to model the system behavior in a later stage of the 
project. Sequence diagrams, EFFBDs, and IDEEO diagrams were developed in order to 
visually describe each system. 

The system evaluation for the current ADAS, Alternative #1, and Alternative #2 
was next. A model was developed in the ExtendSim tool for each system concept. Input 
data and variables were set for each condition so that after a statistically significant 
number of runs (500) in order to generate relevant data to estimate performance of each 
system alternative. Each model was run against a single DRM, which contained the 
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parameters such as number and placement of ordinance dropped on the runway of 
interest, spread of bomblets, and probability of either detonating or becoming classified 
as UXO in the area of interest. 

Cost analysis was done in order to provide the stakeholders with information 
relevant to each design choice. The development costs were the R&D and SE costs, 
which went into the design and construction of the systems. The remaining costs, 
personnel and O&S, were the life cycle cost to keep the systems operational for a 10-year 
time period. This data provided the one-time cost for developing the system of interest, 
and then provided the cost to support the system over a 10-year operational period. 
Although these are not considered significant cost drivers, the loss of a soldier cannot be 
viewed in the same way. It was estimated by an article in the New York Times (Marsh 
2007) that the price for a military life was in the order of $1.7M. This was further broken 
down into $500K for the deceased, and $1.2M for the survivor benefits. Besides the 
military benefits of restoring sortie operations directly after an attack, the additional 
benefit of the lADAS system would be to minimize the risk to personnel performing the 
AD AT role. 

The results of all of the cost modeling and simulation runs can be found 
summarized in Table 41. The values shown reflect the key MOEs for the system and how 
each solution performed against those measures. 


Table 41. Comparison of Results of Cost Modeling and Simulation 

Runs 


MOE 

Threshold 

Objective 

Current 

System 

lADASI 
RPA 

lADAS II 
Tower 

Percent Damage 

Assessed 

90% 

90% 

95% 

97% 

93% 

Percent UXO assessed 

80% 

90% 

95% 

84% 

70% 

Damage Assessment 

Time 

45 min 

30 min 

174 min 

52 min 

47 min 


Total Cost 

$1782K 

$2944K 

$1426K 
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Table 42 describes the overall conditions, comparing the current manual operation 
with those alternatives described in the previous sections of the report. The lADAS 
Alternative Assessment was provided for the stakeholders to have a high-level view of 
the performance differences between the three systems performing ADA activities. The 
merits of each system can be evaluated against each other, and the metrics used to 
determine “success” against a known DRM. A full Doctrine, Organization, Training, 
Materiel, Leadership and Education, Personnel, Facilities and Policy analysis for both 
alternative systems could not be conducted within the time limitations of the project. In 
order to make as complete a comparison as possible, focus was placed on the 
development costs (software, systems engineering, and 10-year maintenance) and 
instantiation costs (hardware, installation, and 10-year maintenance), which could be 
readily accessed in the timeframe for this report. 


Table 42. lADAS Alternative Assessment 


Reference 

Stakeholder Needs 

Met/Not Met 

Current 

System 

Alternative 1 

Alternative 2 

1.0 

Damage Assessment 

95.3% 

96.8% 

93% 

1.1 

lADAS shall detect 

MET 

MET 

MET 


where an airfield has 
been damaged. 




1.2 

lADAS shall 
classify the type of 
airfield damage. 

MET 

MET 

MET 

1.3 

lADAS shall locate 
and measure airfield 
damage. 

MET 

MET 

MET 

2.0 

UXO Assessment 

95.4% 

83.9% 

70.3% 

2.1 

lADAS shall detect 
UXO on airfield 
surfaces. 

MET 

MET 

2.2 

lADAS shall 
classify the type of 
UXO on airfield 
surfaces. 

MET 

MET 


Continued next page. 
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Reference 

Stakeholder Needs 

2.3 

lADAS shall 
provide a location of 
UXO on airfield 
surfaces. 

2.4 

lADAS shall 
automatically transmit 
damage and UXO data 
into GeoExPT. 

2.5 

lADAS shall not 
expose personnel to 
explosive hazards 
during UXO and 
damage assessment 
activities. 

3.0 

Time Assessment 

3.1 

lADAS shall 
complete damage and 
UXO assessment and 
reporting in less than 

30 minutes 

(objective)/45 minutes 
(threshold). 


Table 42.Continued from previous page. 


Met/Not Met 



Alternative 1 


Alternative 2 


MET 


MET 


MET 


52.1 minutes 


MET 


MET 


47.2 minutes 



^Although neither alternative met the threshold time requirement of 45 minutes, both 
alternatives were close and significantly better than the current methodology of the 
current system. The slight differences could be attributed to the resolution of the 
ExtendSim model. 


B. PROJECT CONCLUSIONS 

lADAS I was the clear recommendation to the stakeholders. The system met, or 
exceeded, the threshold values for both assessing the percent damage and UXO mission 
parameters. The assessment time did not met the threshold value (45 minutes) for 
completing the assessment time, but the overall capability of the system delivers to the 
intent of significantly reducing the ADA timeframe. Eor the DRM scenario studied in this 
project, the current ADAT time was estimated at 174 minutes. The lADAS I completed 
the simulation in just 52 minutes. The implementation cost for lADAS I was higher than 
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lADAS II, but the capability and modularity of lADAS I were projected to be 
significantly more valuable. 

lADAS I has a significantly smaller footprint on the airfield of interest and it was 
easily portable to other airfields as required. This would include a small hardened storage 
container for the RPA and spares, along with the ground control station. The I AD AS II 
would be significantly larger, having towers placed at fixed intervals along the area of 
interest. The development cost of lADAS I was significantly larger than that of lADAS 
II, but once the software product has been completed, the system can be adapted rather 
easily to a new set of airfield parameters. This was one of the key benefits of the lADAS 
I alternative. 

Further discussion of the value for lADAS I to both the DOD and industry will be 
continued in the next section of the report. 

C. RECOMMENDATIONS FOR FUTURE STUDIES 

The complete story for both lADAS I and lADAS II were not able to be fully 
developed during the timeframe of this report. There are hundreds of different RPA 
options, and dozens of tower elements that could have been considered. Another 
complicated variable was the day camera system for either system alternative. Each of 
these hardware components could have been a separate investigation in its own right. A 
single alternative was selected and data presented for stakeholder consideration. Future 
studies may want to vary the camera systems and compare image quality vs. system 
effectiveness vs. system cost. As the quality of day camera imagery continues to improve, 
one of the benefits to either lADAS alternatives would be automatic system performance 
improvement as the current cameras are replaced. In addition, as the processing speed of 
computers continues to improve, and network communications continue to speed-up, the 
lADAS alternatives could take advantage of both of these factors. As noted in the 
computations for lADAS I and lADAS II, communication and processing were 
significant contributors to the time spent on the overall mission. Without any 
modifications to the system software, and just migrating to new hardware and 
communication components, the system may meet/exceed the current threshold values set 
in this report. 
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APPENDIX A. PROGRAMS AND TOOLS 


A. EXTENDSIM 

ExtendSim is a computer-based mathematical modeling and simulation tool that is 
used to help predict the behavior or performance on new systems or predict the effect of 
changes on existing systems. Through ExtendSim, any system or process can be 
simulated using a scalable, logical, and easy-to-use format (Imagine That Inc. 2016). 

According to the manufacturer’s website, ExtendSim can perform the following 
functions: 

1. Predict the course and results of certain actions 

2. Gain insight and stimulate creative thinking 

3. Visualize your processes logically or in a virtual environment 

4. Identify problem areas before implementation 

5. Explore potential effects of modifications 

6. Confirm that all variables are known 

7. Optimize operations 

8. Evaluate ideas and identify inefficiencies 

9. Understand why observed events occur 

10. Communicate the integrity and feasibility of your plans (Imagine That Inc. 
2016) 

B. INNOSLATE 

Innoslate is a collaborative online SE tool that provides integrated solutions and is 
capable of providing full life cycle support from requirements definition and management 
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to operations and support (SPEC Innovations 2016). Innoslate supports the following SE 
processes: 

• the Model-Based Systems Engineering (MBSE) using industry standards 
such as systems modeling language (SysME) and IDEEO thus allowing for 
end-to-end design, modeling, and traceability. 

• the Requirements Management by keeping the whole SE team working on 
a centralized version of the document 

• the Department of Defense Architecture Eramework (DoDAE) with an 
easy to use interface. Innoslate will generate the diagrams, matrix, or 
reports. 

• the Configuration Management is established through full version control 
of every entity within the model. 

C. COSYSMO/COCOMOII 

COSYSMO is a model used to estimate the SE effort for large-scale systems and 
includes both hardware and software. COSYSMO identifies many standard SE tasks and 
supports the different life cycle phases. COSYSMO is generally used to support the 
following SE functions: (Massachusetts Institute of Technology 2017) 

• reuse in SE 

• risk modeling in SE 

• the SE schedule estimation 

• the SE sizing 

• cost modeling 

COCOMO II is a model that aids in estimating cost, effort, and schedule for 
software development. COCOMO II is generally used to support the following decision 
points: 


108 



• making investment or other financial decisions involving a software 
development effort 

• setting project budgets and schedules as a basis for planning and control 

• deciding on or negotiating tradeoffs among software cost, schedule, 
functionality, performance or quality factors 

• making software cost and schedule risk management decisions 

• deciding which parts of a software system to develop, reuse, lease, or 
purchase 

• making legacy software decisions such as what parts to modify, phase out, 
or outsource. (Center for Systems and Software Engineering 2017) 
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APPENDIX B. INSTITUTIONAL REVIEW BOARD 

QUESTIONS 

Personnel to be interviewed: Subject matter experts (military and DOD 
civilians) who are responsible for conducting airfield damage assessments as well as 
personnel working on improving current ADA methodology and techniques may be 
interviewed. This may include personnel from Air Damage Assessment Teams (ADATs), 
Military Research Centers, and program personnel familiar with the Rapid Airfield 
Damage Assessment System (RADAS) or other systems in development. 

Questions: 

1. What airfield damage assessment techniques are currently being used by your 
group? 

2. What techniques provide the most accurate results when compared to actual 
airfield damage? 

3. What technique provides the least accurate results when compared to actual 
airfield damage? 

4. What are the tasks which initiate airfield damage assessment and conclude 
airfield damage assessment? 

5. How many personnel are planned for each sub-activity for airfield damage 
assessment? 

6. Are the sub-activities all performed in serial, parallel, or a mix of both? 

7. Are any of the sub-activities prioritized over the other? 

8. What factors increase the time required to conduct airfield damage sub 
activities? 
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9. Does the current airfield assessment methodology allow for flexibility in the 
approach for completing tasks during airfield damage assessment? If so, 
which component of the Damage Assessment Team would make that call? 

10. How does the risk of UXO complicate an airfield damage assessment? 

11. Are there any documents which detail the baseline requirements and/or 
methods for airfield damage assessment? 

12. What is the average time spent conducting the airfield damage assessment? 
What factors affect the time required to complete an airfield damage 
assessment. 

13. What are the baseline (threshold and objective) metrics for airfield damage 
assessment? 

14. In what areas can airfield damage assessment be improved? 

15. Can you describe how the RADAS system differs from the manual method 
currently being used? 

16. Are there any documents outlining the current RADAS system and its 
capabilities? 

17. Is the RADAS system currently fielded and type classified or still in R&D? 

18. What are the baseline (threshold and objective) metrics for the RADAS 
system? How were these derived? 

19. What technologies are being developed that will benefit airfield damage 
assessment? 

20. What factors increase or decrease the difficulty in performing an airfield 
damage assessment? 

21. Has the focus been on improving runway repair time more than improving 
assessment time? If so, why? 
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22. Are the manual assessment procedures similar to what they have been since 
the 1950s? Airfield damage repair has gone through some improvements since 
the 1950s. 


23. What are the main obstacles preventing the Air Force from modernizing the 
assessment procedures? 

24. What is the measure (definition of success) of improvement for airfield 
damage assessment? 
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Responses from Airfield Damage Repair Program Manager 

1. What is the average time spent conducting the airfield damage assessment? 
What factors affect the time required to complete an airfield damage 
assessment. Today with AD AT is takes 2-3 hours. [The] RADAS is expected 
to get it down to 30-45 minutes. 

2. What are the baseline (threshold and objective) metrics for airfield damage 
assessment? See accompanying slide deck. 

3. In what areas can airfield damage assessment be improved? The automated 
damage declaration. 

4. Can you describe how the RADAS system differs from the manual method 
currently being used? RADAS is intended to eliminate the need for Airman on 
the ground. It should an automated system that can feed damage inputs to 
GeoExPT for the MAOS selection. 

5. Are there any documents outlining the current RADAS system and its 
capabilities? See accompanying slide deck. 

6. Is the RADAS system currently fielded and type classified or still in R&D? 
Still in RDT&E. However, GeoExPT is fielded. 

7. What are the baseline (threshold and objective) metrics for the RADAS 
system? How were these derived? See accompanying slide deck. The 
requirements started with PACAE, but were validated with an HPT. 

8. What technologies are being developed that will benefit airfield damage 
assessment? Sensors, platforms, and declaration software. 

9. What factors increase or decrease the difficulty in performing an airfield 
damage assessment? Depends on sensor; but thermal cross over, rain, fog, 
smoke, brass flake. 
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10. Has the focus been on improving runway repair time more than improving 
assessment time? If so, why? We’ve been looking at the holistic problem. Just 
happened that repair was easier to solve. Although it’s A solution, not the 
THE solution. We are still working to implement efficiencies, and ease the 
logistics burden. 

11. Are the manual assessment procedures similar to what they have been since 
the 1950s? Airfield damage repair has gone through some improvements since 
the 1950s. No ADAT is essentially unchanged. We did transition from a 
simple alpha/numeric grid to MGRS. 

12. What are the main obstacles preventing the Air Force from modernizing the 
assessment procedures? Training, frequencies, who can fly an RPA, data 
movement, implementing something on the AF GIG. 

13. What is the measure (definition of success) of improvement for airfield 
damage assessment? Making incremental improvements; see accompanying 
slide deck. 
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Responses from AFCEC/CXD 

1. What airfield damage assessment techniques are currently being used by your 
group? The current methods include visual observation from the air traffic 
control tower and employing an airfield damage assessment team comprised 
of Explosive Ordnance Disposal (EOD) and an Engineering Assistant (EA) 
personnel. Currently we are testing use of Small Unmanned Aerial Systems 
(SUAS) to conduct airfield assessments. 

2. What techniques provide the most accurate results when compared to actual 
airfield damage? The physical run of the airfield provides us the most accurate 
data at this time. However, developing SUAS technologies are proving 
effective. The downside of having EOD/EA personnel run the airfield is the 
amount of time required and the exposure of these personnel to the hazards of 
Unexploded Ordnance (UXO), crater, further enemy attacks. 

3. What technique provides the least accurate results when compared to actual 
airfield damage? The visual observation from the tower or other stand-off 
method from stationary towers using some form of high speed laser 
rangefinder devices. 

4. What are the tasks which initiate airfield damage assessment and conclude 
airfield damage assessment? The tasks associated are to rapidly ascertain the 
amount of damage on the airfield operating surfaces following an enemy 
attack. This damage includes craters, spall fields, surface and buried (holes of 
entry) UXO. The final step is using this data to select the best candidate for a 
Minimum Airfield Operating Strip (MAOS). 

5. How many personnel are planned for each sub-activity for airfield damage 
assessment? The minimum using current methods would be (2) EOD 
technicians, and (1) EA doing the physical Damage Assessment Team (DAT) 
run on the airfield (physical size of airfield may warrant employment of 
additional DAT capability), and another EOD/EA technician in the 


116 



Emergency Operations Center (EOC) to assist in plotting the damage and 
selecting a MAOS. Note: The final selection is a C2 function. 

6. Are the sub-activities all performed in serial, parallel, or a mix of both? The 
sub-activities are performed in parallel. 

7. Are any of the sub-activities prioritized over the other? The most critical ADR 
function (sub-activity) following an attack is completion of the actual airfield 
assessment. The UXO mitigation and repair activities normally do not start 
until the MAOS is determined. Note: This could change if the entire 10,000’ 
X 150’ runway must be cleared based on mission and CONOPs. 

8. What factors increase the time required to conduct airfield damage sub 
activities? The factors that increase the amount of time is physical size of 
airfield, amount of damage on the airfield, weather conditions (night/day), 
smoke clouds from fires/explosions, and number of trained personnel to 
perform sub activities. (Note: The ability to conduct realistic home-station 
training and periodic capstone events is paramount). 

9. Does the current airfield assessment methodology allow for flexibility in the 
approach for completing tasks during airfield damage assessment? If so, 
which component of the Damage Assessment Team would make that call? 
Under current methods, the call would either be made from the ADAT Team 
Eead, normally the senior EOD person, or the EOC function. 

10. How does the risk of UXO complicate an airfield damage assessment? 
Depending on the type of ordnance, and growing threat of Anti-Access/Area 
Denial (A2/AD), the ADAT team is in close proximity to these hazards. This 
drives the need for an armor platform, and remaining undercover, which has 
potential to challenge accurate data collection (size of crater, width/depth 
without physical measurements). 

11. Are there any documents which detail the baseline requirements and/or 

methods for airfield damage assessment? There are baseline documents from 
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large scale exercises such as Salty Demo in 1985, Spangdahlem Air Base in 
Germany. There are other Air Force CE Playbooks, and Tactics, Techniques, 
and Procedures (TTPs) that further define existing baseline requirements. The 
biggest one is establishing a timeline to complete assessment function, 
normally 30-minutes from alarm condition BLACK, and initial release of 
specialized team as directed by LOG C2. 

12. What are the baseline (threshold and objective) metrics for airfield damage 
assessment? The only current base-line I am aware of is the 30-minute time 
requirement. 

13. In what areas can airfield damage assessment be improved? The areas for 
improvement are using advanced technologies to perform this function versus 
physical runs of the airfield operating surfaces. This has great potential to 
reduce time/improve accuracy of collected data. 

14. Can you describe how the RADAS system differs from the manual method 
currently being used? The RADAS incorporates either SUAS platforms or use 
of fixed sensors on the airfield to collect and report damage information. 

15. Are there any documents outlining the current RADAS system and its 
capabilities? These would be maintained within or Acquisition and 
Requirements Division (CXA) within the Air Force Civil Engineer Center 
(AFCEC) Tyndall location. I am unsure if these would be releasable. 

16. Is the RADAS system currently fielded and type classified or still in R&D? I 
believe the system is in RDT&E (use of 3600 dollars), but testing of 
Commercial off the Shelve (COTS) solutions is in progress. The software 
piece is the bigger challenge in my opinion. This includes building a 3D 
digital library of UXO that the software would be able to identify ordnance 
type to some level of accuracy. 

17. What are the baseline (threshold and objective) metrics for the RADAS 

system? How were these derived? I am uncertain of the specific metrics and 
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baselines for RADAS besides the time-line. We have been feeding some 
specific requirements for UXO mitigation. For the immediate solution, the 
goal is to get ordnance type (similar to information in the Airman’s Manual on 
UXO identification charts) and overall numbers. The future developments 
could include color and markings, fuze type by function, etc. 

18. What technologies are being developed that will benefit airfield damage 
assessment? The use of SUAS, reducing exposure of personnel. The use of 
software to assist in the identification of UXO, holes of entry, and surface 
damage to the airfield. Once information is collected, the system will suggest 
MAOS based collected data, and mission need (type of aircraft), ultimate 
selection will still be from a human C2 element. 

19. What factors increase or decrease the difficulty in performing an airfield 
damage assessment? The factors that increase future challenges are cyber 
threats, and jamming, will our SUAS platforms be capable of operating 
without interference, and will we have secure communications between 
RADAS, EOC, UXO mitigation, and repair teams, all of which are critical 
users of this information. In regards to decreasing difficulty, the emerging 
technologies in SUAS, delivering high speed cameras, and innovative 
software that can identify ordnance type, and approximate width/depth of 
craters and spall fields. The ability to run multiple platforms simultaneous on 
the airfield will reduce amount of time to complete assessment function. 

20. Has the focus been on improving runway repair time more than improving 
assessment time? If so, why? The previous focus was more on the repair piece 
of RADR, this was primarily due to the PACAF Advanced Concept and 
Technologies Demonstration (ACTD), which focused on repair, assessment 
and UXO mitigation were not included in this initial ACTD. Due to this, 
repair is several years ahead of assessment/UXO mitigation on development 
of future technologies for the 2035 threat. 


119 



21. Are the manual assessment procedures similar to what they have been since 
the 1950s? Airfield damage repair has gone through some improvements since 
the 1950s. Although, the manual assessment procedures are somewhat similar 
to legacy methods. The differences is using new technologies such as laser 
range finders, better secure communications with EOC and other ADR teams, 
and increased protection of teams using improved armor vehicle platforms 
such as MRAPs and MATVs. 

22. What are the main obstacles preventing the Air Force from modernizing the 
assessment procedures? The main obstacle is technology (although this gap is 
rapidly closing), and future training requirements. The current SUAS program 
requires stringent “pilot” certification and robust licensing requirements. I 
believe we still need to answer if this is an inherent CE capability, or is it a 
broader Air Force requirement that could be performed by rated pilots. The 
bottom line is we are users of the collected data ~ who is responsible for that 
collection is another matter. 

23. What is the measure (definition of success) of improvement for airfield 
damage assessment? The speed and accuracy of the data being collected ~ 
everything under RADR is based on amount of time to recover the airfield and 
begin sortie generation, hence the name “Rapid.” 
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Responses from Executive Officer to the Director of Civil Engineers, Deputy 
Chief of Staff for Logistics, Engineering and Force Protection, Headquarters U.S. 
Air Force, Washington, D.C 

1. What airfield damage assessment techniques are currently being used by your 
group? Our units use two methods in concert 1) Counter Rocket and Mortar 
(CRAM) or similar technology is used to identify the Point of Origin (POO) 
and Point of Impact (POI); and 2) EOD responds to perform a visual 
inspection of the POL Following visual inspection, the EOD team will 
generally immediately transition to UXO mitigation phase (recon, identify, 
render-safe and dispose of UXO) and provide real-time feedback to Engineers 
on any airfield pavement damage that requires repair. 

2. What techniques provide the most accurate results when compared to actual 
airfield damage? Visual inspection of suspected POI using location data 
obtained by CRAM output. 

3. What technique provides the least accurate results when compared to actual 
airfield damage? Visual inspection without POI data. 

4. What are the tasks which initiate airfield damage assessment and conclude 
airfield damage assessment? Airfield Damage Assessment is initiated when 
the CRAM alerts incoming Indirect Fire (IDF.) Following impact, EOD takes 
POI data and responds to impact location. The team conducts a visual 
inspection to recon, identify, render-safe and dispose of ordnance item. 
Following UXO mitigation, engineers respond to conduct expedient repair 
(whether cold asphalt or quick-set concrete patch). 

5. How many personnel are planned for each sub-activity for airfield damage 
assessment? We don’t have a standard sub-activity team size. Typically, two 
EOD technicians (US Army standard team size) respond during damage 
assessment and UXO mitigation phase; 1-5 military or contractor Engineers 
participate in the damage repair phase. 
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6. Are the sub-activities all performed in serial, parallel, or a mix of both? The 
sub-activities (damage assessment, UXO mitigation and damage repair 
phases) occur sequentially. Mobilization can occur at the same time, but the 
steps are performed in serial. 

7. Are any of the sub-activities prioritized over the other? None. 

8. What factors increase the time required to conduct airfield damage sub 
activities? Incomplete or no CRAM data. Additional IDF during the process. 
Darkness. Extreme hot or cold temperatures. 

9. Does the current airfield assessment methodology allow for flexibility in the 
approach for completing tasks during airfield damage assessment? If so, 
which component of the Damage Assessment Team would make that call? 
The current methodology used at airbases in Afghanistan is a flexible and 
well-rehearsed process. Following CRAM output of POI, base recovery 
decision makers (Senior Airfield Authority or designee) can make 
assessments of whether to shut down the airfield. 

10. How does the risk of UXO complicate an airfield damage assessment? 
Because of the risk of UXO, EOD always participates in airfield damage 
assessment. This is not a complicating factor unless multiple events require 
prioritization of EOD teams to multiple locations on and off the airfield. 

11. Are there any documents which detail the baseline requirements and/or 
methods for airfield damage assessment? Unknown whether our subordinate 
units use specific documents to detail baseline requirements and/or methods. I 
assume that TOC/BDOCs are using checklists. 

12. What is the average time spent conducting the airfield damage assessment? 

What factors affect the time required to complete an airfield damage 

assessment. With reliable CRAM data, airfield damage assessment phase can 

be done relatively quickly (15 minutes +!-). [The] UXO mitigation phase is 

depending on the type of UXO and condition it’s found. Eikewise, airfield 
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repair phase is depending on the type and severity of damage to pavements 
and underlying soil structures. 

13. What are the baseline (threshold and objective) metrics for airfield damage 
assessment? Unknown whether our subordinate units use threshold and 
objective metrics for airfield damage assessment. Because they are conducting 
real world combat missions from the airfields, the objective is likely to 
minimize runway down time to limit/eliminate effects on ATO sortie 
generation. 

14. In what areas can airfield damage assessment be improved? Outside of threats 
commonly found in CJOA-A (primarily IDF), I believe airfield damage 
assessment needs to be improved in order to combine multiple real time data 
sources (CRAM or other radar, visual, tower cameras, UAVs, etc.) to identify 
multiple points of impact. This real time data can be combined with remote 
assessment techniques (CROWs, tower cameras, UAVs) to build a UXO 
mitigation phase plan of attack. 

15. Can you describe how the RADAS system differs from the manual method 
currently being used? RADAS aims to combine many of the real time data 
sources I referred to above to provide situational awareness following an 
attack. This is different than the current manual method in that it does not 
require a manual approach to build initial situational awareness of UXO and 
damage locations. 

16. Are there any documents outlining the current RADAS system and its 
capabilities? Refer you to AFCEC/CX for more information. 

17. Is the RADAS system currently fielded and type classified or still in R&D? 
Refer you to AFCEC/CX for more information. 

18. What are the baseline (threshold and objective) metrics for the RADAS 
system? How were these derived? Refer you to AECEC/CX for more 
information. 
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19. What technologies are being developed that will benefit airfield damage 
assessment? Refer you to AFCEC/CX, IHEODTD and sister-service research 
efforts for more information. Anecdotally I have heard that U.S. Navy 
engineers are also doing research in the area of airfield damage assessment 
and repair. 

20. What factors increase or decrease the difficulty in performing an airfield 
damage assessment? Incomplete real-time data (remote visual, radar, etc.). 
Repeat attacks halting/delaying efforts. Weather. Equipment malfunctions. 
Eow visibility/hours of darkness. 

21. Has the focus been on improving runway repair time more than improving 
assessment time? If so, why? Refer question to AECEC/CX. I don’t have a 
complete sight picture on how AECEC has expended time and resources in the 
runway repair phase vs. the assessment or UXO mitigation phase. 

22. Are the manual assessment procedures similar to what they have been since 
the 1950s? Airfield damage repair has gone through some improvements since 
the 1950s. Refer question to AECEC/CX. I don’t have a complete historical 
sight picture on TTPs currently taught in SIEVER EEAG when compared to 
those techniques taught in the 1950s. 

23. What are the main obstacles preventing the Air Eorce from modernizing the 
assessment procedures? I do not have current and firsthand knowledge to 
answer definitively, but I assume technology readiness levels and resourcing 
present challenges to modernization of the overall Airfield Damage 
Assessment program. 

24. What is the measure (definition of success) of improvement for airfield 
damage assessment? In my opinion, the definition of success should be the 
proven capability to conduct Airfield Damage Repair (of which airfield 
damage assessment is a sub-task) in the time frames established by combatant 
commanders, joint force commanders and combined/joint force air component 
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commanders. These time frames are aggressive but not publically releasable 
due to classification level. 
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APPENDIX C. ADDITIONAL DATA 


Graphical representations of the DRM weapons are shown in Figures C-1 through 


C-3. 


DRM Weapon Information 


• Russian RBK-500 BetAB Cluster Bomb 

• Anti-runway Cluster-Bomb Unit (CBU) intended to penetrate and damage 
concrete structures and runways 

• Characteristics: 

• Carries 12 BetAB sub munitions (fragmentation) 

• Each BetAB has a crater size of 4 m''2 

• Maximum BetAB dispersal diameter = 40 m 

• Assumed dud rate of 16% (based upon UXO reports) 



Figure C-1. Russian RBK-500 BetAB Cluster Bomb. Sources: Jane’s 
Air Launched Weapons (2007) and Flankers Site (2017). 
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DRM Weapon Information 

• Russian RBK-500RTM AO Cluster Bomb 

• Anti-personnel Cluster-Bomb Unit (CBU) intended to spread multiple sub¬ 
munitions over a wide area in order to prevent the use of an area 

• Characteristics; 

• Carries 108 AO-2.5RT sub munitions (fragmentation) 

• Has a footprint of 224,000 sq feet (267 foot radius) 

• Assumed dud rate of 16% (based upon UXO reports) 



Figure C-2. Russian RBK-500RTM AO Cluster Bomb. 
Sources: International Campaign to Ban Landmines (2012) and 
The Fighter Collection & Eagle Dynamics (2013) 
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DRM Weapon Information 

• Russian FAB-500 T General Purpose Bomb 

• Air-dropped bomb intended to inflict damage to the runway through blast, 
fragmentation, and penetration through the runway. 

• Effects; 

• Crater Depth = 13 m (43 feet) 

• Crater Diameter = 22.5 m (74 feet) 

• Clean up/Spalling Diameter = 430 m (1411 feet) 



Figure C-3. Russian FAB-500T. Source: Global Security.org 

(2016) 
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Table C-1. DRM RBK-500 BetAB (1) Pattern 



X 

Y 



X 



27R/9L 

9R/27L 


CBU Target 

3260.5 

700 


Radius 

65 


Total 

Hits 

12 

0 









Total 

UXO 

4 

0 


Bomblet 

uxo 

X location 

Y location 

Hit 27R/ 
9L 

Hit 9R/ 
27L 

27R/9L 

UXO 

9L/27R 

UXO 


Radius 

Angle 

1 

FALSE 

3252.26 

692.34 

TRUE 

EALSE 

EALSE 

EALSE 


11.25 

3.89 

2 

TRUE 

3252.55 

687.78 

TRUE 

EALSE 

TRUE 

EALSE 


14.58 

4.14 

3 

EALSE 

3252.07 

704.17 

TRUE 

EALSE 

EALSE 

EALSE 


9.41 

2.68 

4 

EALSE 

3261.89 

710.38 

TRUE 

EALSE 

EALSE 

EALSE 


10.47 

1.44 

5 

EALSE 

3306.73 

694.67 

TRUE 

EALSE 

EALSE 

EALSE 


46.53 

6.17 

6 

TRUE 

3236.80 

692.55 

TRUE 

EALSE 

TRUE 

EALSE 


24.85 

3.45 

7 

EALSE 

3255.50 

655.04 

TRUE 

EALSE 

EALSE 

EALSE 


45.23 

4.60 

8 

TRUE 

3259.95 

693.96 

TRUE 

EALSE 

TRUE 

EALSE 


6.07 

4.62 

9 

EALSE 

3268.00 

692.00 

TRUE 

EALSE 

EALSE 

EALSE 


10.97 

5.47 

10 

EALSE 

3302.93 

719.51 

TRUE 

EALSE 

EALSE 

EALSE 


46.70 

0.43 

11 

EALSE 

3270.92 

710.41 

TRUE 

EALSE 

EALSE 

EALSE 


14.73 

0.79 

12 

TRUE 

3261.97 

717.67 

TRUE 

EALSE 

TRUE 

EALSE 


17.73 

1.49 
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Table C-2. DRM RBK-500 BetAB (2) Pattern 



X 

Y 



X 



27R/9L 

9R/27L 


CBU Target 

1000 

75 


Radius 

65 


Total 

Hits 

0 

12 









Total 

UXO 

0 

5 













Bomblet 

uxo 

X location 

Y location 

Hit 27R/ 
9L 

Hit 9R/ 
27L 

27R/9L 

UXO 

9L/27R 

UXO 


Radius 

Angle 

1 

FALSE 

1000.44 

74.81 

FALSE 

TRUE 

FALSE 

FALSE 


0.48 

5.88 

2 

FALSE 

953.02 

72.45 

FALSE 

TRUE 

FALSE 

FALSE 


47.05 

3.20 

3 

FALSE 

979.24 

128.38 

FALSE 

TRUE 

FALSE 

FALSE 


57.27 

1.94 

4 

FALSE 

970.28 

115.30 

FALSE 

TRUE 

FALSE 

FALSE 


50.07 

2.21 

5 

TRUE 

997.13 

72.58 

FALSE 

TRUE 

FALSE 

TRUE 


3.76 

3.84 

6 

FALSE 

1012.61 

123.41 

FALSE 

TRUE 

FALSE 

FALSE 


50.03 

1.32 

7 

TRUE 

988.16 

82.63 

FALSE 

TRUE 

FALSE 

TRUE 


14.09 

2.57 

8 

TRUE 

992.42 

53.89 

FALSE 

TRUE 

FALSE 

TRUE 


22.43 

4.37 

9 

TRUE 

978.44 

64.64 

FALSE 

TRUE 

FALSE 

TRUE 


23.92 

3.59 

10 

FALSE 

985.21 

81.10 

FALSE 

TRUE 

FALSE 

FALSE 


16.00 

2.75 

11 

TRUE 

1057.03 

77.03 

FALSE 

TRUE 

FALSE 

TRUE 


57.06 

0.04 

12 

FALSE 

1015.43 

45.23 

FALSE 

TRUE 

FALSE 

FALSE 


33.53 

5.19 
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Table C-3. DRM RBK-500 AO-2.5.RT 



X 

Y 



X 






27R/9L 

9R/27L 

CBU 

Target 

500 

700 


Radius 

267 





Total Hits 

43 

0 











Total 

UXO 

4 

0 














Bomblet 

UXO 

X 

location 

Y location 

Hit 27R/ 
9L 

Hit 9R/ 
27L 

27R/9L 

UXO 

9L/27R 

UXO 


Radius 

Angle 



1 

FALSE 

501.8957 

735.3605 

TRUE 

FALSE 

FALSE 

FALSE 


35.4112 

1.51726 



2 

FALSE 

482.6544 

731.2462 

TRUE 

FALSE 

FALSE 

FALSE 


35.7378 

2.0776 



3 

FALSE 

580.6993 

686.38194 

TRUE 

FALSE 

FALSE 

FALSE 


81.84023 

6.1160 



4 

FALSE 

444.2820 

768.9314 

FALSE 

FALSE 

FALSE 

FALSE 


88.6343 

2.2506 



5 

FALSE 

415.3308 

634.4573 

FALSE 

FALSE 

FALSE 

FALSE 


107.0734 

3.8003 



6 

FALSE 

348.3067 

484.21889 

FALSE 

FALSE 

FALSE 

FALSE 


263.7657 

4.0997 



7 

FALSE 

697.6822 

614.3782 

FALSE 

FALSE 

FALSE 

FALSE 


215.42831 

5.8744 



8 

FALSE 

482.8694 

495.7018 

FALSE 

FALSE 

FALSE 

FALSE 


205.0151 

4.6287 



9 

FALSE 

486.4539 

688.4423 

TRUE 

FALSE 

FALSE 

FALSE 


17.8067 

3.8479 



10 

FALSE 

335.1401 

748.2649 

TRUE 

FALSE 

FALSE 

FALSE 


171.7797 

2.8568 



11 

FALSE 

445.8302 

725.8789 

TRUE 

FALSE 

FALSE 

FALSE 


60.0341 

2.6959 



12 

FALSE 

476.4532 

710.1315 

TRUE 

FALSE 

FALSE 

FALSE 


25.6339 

2.7353 



13 

FALSE 

496.8794 

703.9253 

TRUE 

FALSE 

FALSE 

FALSE 


5.0146 

2.2425 



14 

FALSE 

480.4017 

706.4275 

TRUE 

FALSE 

FALSE 

FALSE 


20.6254 

2.8247 



15 

FALSE 

530.6377 

719.7591 

TRUE 

FALSE 

FALSE 

FALSE 


36.4567 

0.5728 



16 

FALSE 

383.2987 

819.0994 

FALSE 

FALSE 

FALSE 

FALSE 


166.7449 

2.3460 



17 

FALSE 

600.2355 

513.8571 

FALSE 

FALSE 

FALSE 

FALSE 


211.4151 

5.20635 



18 

FALSE 

458.2637 

614.3516 

FALSE 

FALSE 

FALSE 

FALSE 


95.2763 

4.25896 



19 

FALSE 

536.3748 

925.92 

FALSE 

FALSE 

FALSE 

FALSE 


228.8296 

1.41116 



20 

FALSE 

499.9493 

702.0313 

TRUE 

FALSE 

FALSE 

FALSE 


2.032 

1.5957 



21 

FALSE 

369.5659 

496.5683 

FALSE 

FALSE 

FALSE 

FALSE 


241.6558 

4.14225 




Continued next page. 
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Table C-3. Continued from previous page. 


22 

FALSE 

500.7656 

690.9322 

TRUE 

FALSE 

FALSE 

FALSE 


9.1001 

4.7966 



23 

FALSE 

427.1609 

661.6576 

TRUE 

FALSE 

FALSE 

FALSE 


82.31449 

3.6261 



24 

FALSE 

396.5942 

682.5180 

TRUE 

FALSE 

FALSE 

FALSE 


104.8732 

3.3091 



25 

FALSE 

513.5728 

823.4536 

FALSE 

FALSE 

FALSE 

FALSE 


124.1975 

1.4613 



26 

FALSE 

554.5636 

497.2172 

FALSE 

FALSE 

FALSE 

FALSE 


209.9954 

4.9752 



27 

FALSE 

500.9947 

896.752 

FALSE 

FALSE 

FALSE 

FALSE 


196.7545 

1.5657 



28 

FALSE 

454.3521 

791.4894 

FALSE 

FALSE 

FALSE 

FALSE 


102.245 

2.0336 



29 

FALSE 

601.0013 

540.2267 

FALSE 

FALSE 

FALSE 

FALSE 


189.0205 

5.2761 



30 

TRUE 

545.7199 

730.4717 

TRUE 

FALSE 

TRUE 

FALSE 


54.944 

0.5879 



31 

FALSE 

519.7072 

818.5168 

FALSE 

FALSE 

FALSE 

FALSE 


120.1441 

1.4060 



32 

FALSE 

239.4548 

710.7863 

TRUE 

FALSE 

FALSE 

FALSE 


260.7684 

3.1002 



33 

FALSE 

653.2224 

832.1566 

FALSE 

FALSE 

FALSE 

FALSE 


202.3424 

0.7117 



34 

FALSE 

403.5577 

789.8885 

FALSE 

FALSE 

FALSE 

FALSE 


131.8372 

2.3914 



35 

FALSE 

425.4041 

736.8119 

TRUE 

FALSE 

FALSE 

FALSE 


83.1845 

2.6832 



36 

TRUE 

499.5708 

941.8778 

FALSE 

FALSE 

FALSE 

FALSE 


241.8782 

1.5726 



37 

FALSE 

595.7699 

678.1226 

TRUE 

FALSE 

FALSE 

FALSE 


98.2369 

6.0586 



38 

FALSE 

564.225 

838.5785 

FALSE 

FALSE 

FALSE 

FALSE 


152.7378 

1.1369 



39 

FALSE 

442.04 

719.1564 

TRUE 

FALSE 

FALSE 

FALSE 


61.0375 

2.8224 



40 

FALSE 

352.472 

614.2428 

FALSE 

FALSE 

FALSE 

FALSE 


170.6424 

3.6681 



41 

FALSE 

459.7746 

771.5350 

FALSE 

FALSE 

FALSE 

FALSE 


82.0692 

2.0830 



42 

TRUE 

534.9869 

704.9717 

TRUE 

FALSE 

TRUE 

FALSE 


35.3384 

0.1412 



43 

FALSE 

556.6560 

618.5961 

FALSE 

FALSE 

FALSE 

FALSE 


99.1791 

5.3204 



44 

FALSE 

597.3418 

643.8385 

FALSE 

FALSE 

FALSE 

FALSE 


112.3812 

5.7599 



45 

FALSE 

403.6056 

747.0471 

TRUE 

FALSE 

FALSE 

FALSE 


107.2629 

2.6876 



46 

FALSE 

504.8777 

775.1587 

FALSE 

FALSE 

FALSE 

FALSE 


75.3168 

1.50684 



47 

FALSE 

498.4391 

835.0429 

FALSE 

FALSE 

FALSE 

FALSE 


135.0519 

1.5824 



48 

FALSE 

544.6467 

547.5593 

FALSE 

FALSE 

FALSE 

FALSE 


158.8442 

4.9973 



49 

TRUE 

726.0692 

750.1483 

FALSE 

FALSE 

FALSE 

FALSE 


231.5645 

0.2183 




Continued next page. 


133 




Table C-3. Continued from previous page. 


50 

FALSE 

478.1377 

757.3886 

EALSE 

EALSE 

EALSE 

EALSE 


61.4118 

1.9348 



51 

FALSE 

532.1559 

668.8036 

TRUE 

EALSE 

EALSE 

EALSE 


44.8021 

5.5129 



52 

FALSE 

441.1467 

461.5787 

EALSE 

EALSE 

EALSE 

EALSE 


245.5778 

4.4704 



53 

EALSE 

303.7576 

671.9258 

TRUE 

EALSE 

EALSE 

EALSE 


198.2404 

3.2837 



54 

TRUE 

490.0124 

459.9954 

FALSE 

EALSE 

EALSE 

EALSE 


240.2123 

4.6708 



55 

EALSE 

739.2747 

672.5469 

TRUE 

EALSE 

EALSE 

EALSE 


240.8445 

6.169 



56 

EALSE 

471.8054 

726.3382 

TRUE 

EALSE 

EALSE 

EALSE 


38.5828 

2.3902 



57 

EALSE 

470.1147 

803.6697 

EALSE 

EALSE 

EALSE 

EALSE 


107.8913 

1.8515 



58 

TRUE 

505.2713 

699.2755 

TRUE 

EALSE 

TRUE 

EALSE 


5.3208 

6.1466 



59 

EALSE 

551.9340 

704.4523 

TRUE 

EALSE 

EALSE 

EALSE 


52.1245 

0.0855 



60 

TRUE 

607.2198 

535.1757 

FALSE 

EALSE 

EALSE 

EALSE 


196.6295 

5.2891 



61 

EALSE 

383.1199 

822.2814 

EALSE 

EALSE 

EALSE 

EALSE 


169.1559 

2.3336 



62 

EALSE 

669.1246 

656.1017 

TRUE 

EALSE 

EALSE 

EALSE 


174.729 

6.0292 



63 

EALSE 

429.7875 

676.3861 

TRUE 

EALSE 

EALSE 

EALSE 


74.07711 

3.4660 



64 

EALSE 

487.1729 

685.2889 

TRUE 

EALSE 

EALSE 

EALSE 


19.5179 

3.9953 



65 

EALSE 

476.8386 

677.7383 

TRUE 

EALSE 

EALSE 

EALSE 


32.1253 

3.9072 



66 

EALSE 

468.7925 

799.2154 

EALSE 

EALSE 

EALSE 

EALSE 


104.0077 

1.87554 



67 

TRUE 

393.1081 

821.8579 

FALSE 

EALSE 

EALSE 

EALSE 


162.0964 

2.2909 



68 

TRUE 

571.9861 

621.9643 

FALSE 

EALSE 

EALSE 

EALSE 


106.1677 

5.4575 



69 

EALSE 

511.2804 

442.0496 

EALSE 

EALSE 

EALSE 

EALSE 


258.1969 

4.7561 



70 

EALSE 

558.9692 

586.9819 

EALSE 

EALSE 

EALSE 

EALSE 


127.4773 

5.1933 



71 

EALSE 

488.9730 

620.9322 

EALSE 

EALSE 

EALSE 

EALSE 


79.8331 

4.5738 



72 

EALSE 

557.8391 

760.1503 

EALSE 

EALSE 

EALSE 

EALSE 


83.4472 

0.805 



73 

EALSE 

434.3834 

903.8431 

EALSE 

EALSE 

EALSE 

EALSE 


214.1438 

1.8822 



74 

EALSE 

644.9380 

501.71634 

EALSE 

EALSE 

EALSE 

EALSE 


245.6083 

5.3436 



75 

EALSE 

346.5076 

844.8829 

EALSE 

EALSE 

EALSE 

EALSE 


211.0710 

2.3850 



76 

EALSE 

510.7632 

478.66504 

EALSE 

EALSE 

EALSE 

EALSE 


221.5965 

4.761 
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77 

FALSE 

443.3549 

681.0922 

TRUE 

EALSE 

EALSE 

EALSE 


59.7174 

3.4638 



78 

FALSE 

653.7993 

805.1358 

FALSE 

FALSE 

EALSE 

EALSE 


186.3002 

0.5997 



79 

TRUE 

444.0708 

699.2562 

TRUE 

EALSE 

TRUE 

EALSE 


55.9342 

3.1549 



80 

FALSE 

428.8799 

694.7451 

TRUE 

EALSE 

EALSE 

EALSE 


71.314 

3.2153 



81 

FALSE 

701.7690 

838.5802 

FALSE 

FALSE 

EALSE 

EALSE 


244.7758 

0.6018 



82 

FALSE 

355.2583 

809.5912 

FALSE 

FALSE 

EALSE 

EALSE 


181.55 

2.4935 



83 

FALSE 

583.2062 

732.6377 

TRUE 

EALSE 

EALSE 

EALSE 


89.3784 

0.3738 



84 

FALSE 

644.8251 

819.4775 

FALSE 

FALSE 

EALSE 

EALSE 


187.7476 

0.6898 



85 

FALSE 

391.6471 

465.8855 

FALSE 

FALSE 

EALSE 

EALSE 


257.97278 

4.2789 



86 

FALSE 

577.3118 

665.4725 

TRUE 

EALSE 

EALSE 

EALSE 


84.6714 

5.8632 



87 

FALSE 

558.1704 

800.8978 

FALSE 

FALSE 

EALSE 

EALSE 


116.4653 

1.0478 



88 

FALSE 

562.8236 

764.0383 

FALSE 

FALSE 

EALSE 

EALSE 


89.7091 

0.795 



89 

FALSE 

513.9656 

741.4113 

TRUE 

EALSE 

EALSE 

EALSE 


43.70281 

1.2455 



90 

FALSE 

610.3964 

738.6799 

TRUE 

EALSE 

EALSE 

EALSE 


116.9765 

0.3370 



91 

FALSE 

492.5226 

757.4785 

FALSE 

FALSE 

EALSE 

EALSE 


57.9628 

1.7002 



92 

FALSE 

483.4872 

748.5778 

TRUE 

EALSE 

EALSE 

EALSE 


51.3077 

1.8985 



93 

FALSE 

630.9781 

735.4820 

TRUE 

EALSE 

EALSE 

EALSE 


135.6990 

0.2646 



94 

FALSE 

624.3331 

478.6027 

FALSE 

FALSE 

EALSE 

EALSE 


253.9202 

5.2241 



95 

FALSE 

457.4431 

779.4127 

FALSE 

FALSE 

EALSE 

EALSE 


90.0970 

2.0627 



96 

FALSE 

463.5419 

815.69845 

FALSE 

FALSE 

EALSE 

EALSE 


121.3068 

1.8761 



97 

FALSE 

499.306 

705.0742 

TRUE 

FALSE 

FALSE 

EALSE 


5.1215 

1.7067 



98 

FALSE 

475.0920 

596.9489 

FALSE 

FALSE 

FALSE 

EALSE 


106.0187 

4.4752 



99 

FALSE 

386.1351 

812.7207 

FALSE 

FALSE 

FALSE 

EALSE 


160.2222 

2.3612 



100 

FALSE 

519.5824 

459.0308 

FALSE 

FALSE 

FALSE 

EALSE 


241.7636 

4.7935 



101 

FALSE 

345.4347 

579.0331 

FALSE 

FALSE 

FALSE 

EALSE 


196.2739 

3.8056 



102 

FALSE 

625.1924 

688.6215 

TRUE 

FALSE 

FALSE 

EALSE 


125.7085 

6.1925 



103 

TRUE 

461.7902 

831.6925 

EALSE 

FALSE 

FALSE 

EALSE 


137.1237 

1.8532 
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104 

FALSE 

504.4389 

465.2151 

EALSE 

FALSE 

FALSE 

EALSE 


234.8269 

4.7313 



105 

FALSE 

243.5919 

634.9461 

EALSE 

FALSE 

FALSE 

EALSE 


264.5319 

3.39001 



106 

FALSE 

361.1039 

887.0751 

EALSE 

FALSE 

FALSE 

EALSE 


233.0005 

2.20945 



107 

FALSE 

325.638 

714.0523 

TRUE 

FALSE 

FALSE 

EALSE 


174.9273 

3.0612 



108 

FALSE 

394.9736 

851.7969 

EALSE 

FALSE 

FALSE 

EALSE 


184.5883 

2.1761 




Figures C-4 through C-6 show the ADAS ExtendSim model, simulation inputs and outputs in more detail. 
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Figure C-5. ADAS Simulation Inputs 
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Simulation Outputs (Last Run Only) 


% Damage Detected 

1 

% UXO Detected 

’1 ^ 

ADA Time 

54.47 ‘ 

Launch and Travel time to Start of Survey (min) 

7.478' 

Image Collecton Time (min) 

8.525'^ 

Imagery Processing (min) 

15.244 

Automatic Detection/ClassificatioaTileasurement (min) 

4.4734 

Auto Detect Verification (min) 

4.00744 

Manual Review of Imagery (min) 

4.9351 

Manual Classification (time) 

4.868 

Manual Measurment (time) 

4.937 


Figure C-6. ADAS Simulation Outputs (Last Run Only) 
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Figures C-7 through C-9 show the lADAS I ExtendSim model, simulation inputs and outputs in more detail 
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Simulation Inputs 
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IRPA Camera & UAS Parameters 
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Figure C-8. lADAS I Simulation Inputs 
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Simulation Outputs (Last Run Only) 


% Damage Detected 

1 

% UXO Detected 

'l 

ADA Time 

54.47' 

Launch and Travel time to Start of Survey (min) 

[7.478 

Image Collecton Time (min) 

'8.525" 

Imagery Processing (min) 

15.244 

Automatic Detection.'ClassificstioaT«i1easurement (min) 

4.4734 

Auto Detect Verification (min) 

4.00744 

Manual Review of Imagery (min) 

4.9351 

Manual Classification (time) 

4.868 

Manual Measurment (time) 

4937 


Figure C-9. lADAS I Simulation Outputs (Last Run Only) 


Some of the screen shots for lADAS I & lADAS II alternatives are provided 
below. The first image is from the systems engineering estimate for lADAS I in Figures 
C-lOand C-11. 
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System Cost Model Suite 
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Figure C-10. lADAS I Systems Engineering Costs, Page 1 
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Results 

Systems Engineering 
Effort =21.7 Person-months 
Schedule = 4.1 Months 
Cost = S216926 

Total Size =108 Equivalent Nominal Requirements 


Acquisition Effort Distribution (Person-Months) 
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Figure C-11. lADAS I Systems Engineering Costs, Page 2 


The next image is from the software engineering estimate for lADAS I in Figures 
C-12and C-13. 
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Figure C-12. lADAS I Software Engineering Cost, Page 1 
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Figure C-13. lADAS I Software Engineering Cost, Page 2 
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Figures C-14 through C-16 show the lADAS II ExtendSim model, simulation inputs and outputs in more detail 
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Figure C-15. lADAS II Simulation Inputs 
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Figure C-16. lADAS II Simulation Outputs (Last Run Only) 


Moving to the lADAS II alternative, the next image is from the Systems 
Engineering estimate in Figure C-17. 
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Figure C-17. lADAS II Systems Engineering Cost 
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The final COCOMO model for lADAS II is the software engineering costs in 
Figures C-I8 and C-I9. 
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